To determine if a piece is relevant, you may try to answer the following questions:
Think that you will rarely find a source that provides a complete answer to your main questions and sub-questions and that gives a report of the exact same research or problem you are working on. Thank goodness, otherwise your work would have no use at all....
The crux of science lies in the extent to which an author/researcher performs his work objectively and makes it verifiable. In determining the quality and scientific nature of sources you may start from three kinds of checks:
The selection criteria of the database or search engine you use can give you more certainty about the basic quality of the sources you found. But what criteria do the makers of these databases use?
Citation databases: strictly scientific/scholarly
A very strict database is Web of Science. The selection proces of new journals at Web of Science is partly based on citation data: are there many references to the new journal in the journals already included in the database? In addition the editorial quality and the fixed publication frequency is an important criterion. The inclusion criteria of Scopus are also strict, but the selection relies more on the the advice of a 'content selection & advisory board' than on citation data. For instance Scopus looks at the geographical spread of editors and authors of a journal. Both Web of Science and Scopus make peer review a condition that must be met. With these databases getting included may take a journal several years..
Subject related databases: scientific/scholarly
Subject related databases (for instance Business Source Premier, (Ebsco) Hospitality and tourism Complete (Ebsco), Lieusre and Tourism Databsae (CABI) try to make searchable as many subjects as possible in a discipine. Almost all of them include articles, but there is an enormous difference among the databases when it comes to the inclusion of books, reports, conference proceedings or MA theses. A (small) part of the sources you will find here will not be peer reviewed.
Google Scholar: scientific/scholarly but with 'pollution'
Google Scholar's selection criteria are fairly broad and are partly technical by nature. Google Scholar hardly offers anything to go on as far as scientific quality is concerned. It is true that much material from strictly scientific journals can be found here, but Google Scholar also automatically recognises texts on the web which are probably not scientific, such as articles published on the website of the author or on a university website. That is why you will also find non-peer reviewed material, concept versions, library guides, students' papers etc. Also books from Google Books are added automatically which are recognised as scientific.
Library catalogue: sources relevant to your courses
In library catalogue you will find a wide range of material. Here the main selection criterion is whether something is relevant to education and research of an academy. So a library catalogue may also contain popular-scientific material and also non-scientific sources. In addition to the catalogue BUAS has a broader Metasearch.
Meta databases and meta catalogues: all kinds and levels
Large hybrid search systems such as BUAS' Metasearch, the Dutch Picarta and worldwide WorldCat offer the less to go on due to the many kinds of sources and databases from which they combine all kinds of material. In particular at Picarta and WorldCat because all sources of public libraries are indirectly included there is absolutely no certainty about the scientific level.
http://webform1.nhtv.nl/mediatheek/Link.aspx?ID=2999You can derive the scholarly nature (but not automatically also the quality!) of journal articles from the journals in which they are published. If all is well, you will find the applied criteria in the journal itself or on the website of the journal. Many scientific journals check these via the so-called peer review process. To this end, the article is blindly presented to experts invited by the editors (may be worldwide). In this way the journal guarantees a kind of minimal scientific quality. But it does not mean that articles from peer reviewed journals are always better thanarticles from journals without peer review.
If a (modern) journal is included in Web of Science or Scopus, you may expect that it is peer reviewed. In all other cases you will have to check it yourself on the site of the journal or in the journal list.
Sometimes journals make high demands on articles and some of them reject more than 90 percent of the submitted articles, a large part already before the process of peer review. This rejection rate may even give a journal more status.
Demands a journal makes are always with regard to originality and the interest to the (continuation of) science. In addition there are many extra requirements, such as a clear argument, language and the right references. Sometimes it is demanded that data are made public.
But be careful: also seriously scientific journals contain non-scientific parts such as letters to the editor, book reviews, news, comments, opinion pages and editorials. These parts are often exluded from the peer review check. As a result, you will have to treat them differently when you use them.
Although the subjects written about can be the same in all periodicals (climate change, the financial crisis, popular music) there are considerable differences between journals. You use scientific journals to find out if your subject has already been researched and to cite from. Professional journals are also used to make yourself familiar with your discipline and its organisation, and sometimes also to cite from. General news magazines provide information about social, economic and political issues. Popular journals mainly focus on entertainment.
| Scientific journal | Professional journal | Opinion magazine | Popular journal | |
| Authors |
Always mentioned, always scholars/scientists, usually with work address, editors only take care of introduction |
Usually mentioned, usually scholars/scientists, editors sometimes take care of introduction | Usually mentioned, sometimes anonymous, many articles written by the editorial staff | Usually mentioned, sometimes anonymous, very many articles written by the editorial staff |
| Payment | Editors and authors work for free, editorial board paid professional(s) | core editors are paid, authors work for free | editors are paid, articles by others are also paid for | core editors are paid, articles by freelancers are also paid for |
| Sources | Almost always literature references > 10 | Sometimes literature references < 5 | usually no literature references | usually no literature references |
| Quality check | Peer review | Editors | Editors | Editors |
| Type of content | Research results, analysis, discussion, lots of jargon | News, trends in the professional field, shorter analytical pieces | News, interviews, background, discussion, politics | reports, pictorial articles |
| Typical length of article | 2.500-10.000 words | 400-2.000 words | 500-1.500 words | 300-1.200 words |
| Page numbering | Continues throughout the volume | sometimes continuous, usually for each separate issue | for each separate issue | for each separate issue |
| Design | Simple cover, few pictures, many diagrams, tables, hardly advertisements, many rules about the structure and layout of the article | Relatively cheap design, relatively many advertisements | many news pictures and pictures of persons, many advertisements | colourful, many pictures |
|
Typical publication frequency |
4-10/y |
8-12/y |
26-52/y |
12-52/y |
|
Examples |
PLOS One, Environment & Planning. Tourism Management, Annals of Tourism Research American Economic Review |
Economisch Statistische Berichten, Travel Magazine, Geografie |
Economist, Groene Amsterdammer, Elsevier, Vrij Nederland, Opzij, Time, Der Spiegel |
National Geographic, Kijk, Avantgarde, Avenue, Elle, Quest, OOR |
Some professional journals focus on bringing a discipline to the attention of a large audience and as a result share many characteristics of popular journals. Examples: Historisch Nieuwsblad, Filosofie Magazine, Psychologie Magazine.
According to many, peer review, the system by which fellow experts judge the quality of a piece before publication is one of the pillars of science. Editors can never have enough expertise to judge submitted articles or books. Experts can. Without them knowing the identity of the author, they assess whether the piece meets the scientific norms: stating where the data originate from, a logical analysis, references to relevant sources. Besides they say what the article has to contribute to science. Peer reviewing an article is more about the question of a piece being written according to a good scientific method than about reviewers agreeing with the content. The advice to the editors will be either publishable, publishable with certain adjustments/ not publishable.
The peer rewiew system is subject to much criticism, although many people say that is the 'best worst system' we have. Criticism focusses on:
There are sites where authors share their experiences with the peer review of journals. The Dutch initiative SciRev is one of them.
Some journals, such as PLOS One (Public Library of Science One) are experimenting with new forms of peer review. Peer reviewers only indicate if the articles are written in the right method. Assessing the scientific interest is done at a later stage, by the readers. The use of this method leads to a speedier publication of many articles.
Publication cultures of disciplines differ:
The number of authors that have worked on the article is much higher in the Medical and Natural Sciences than in the Social Sciences and in Humanities. In extreme cases the article carries more than a thousand names of authors
In many disciplines reports are a rich, up-to-date and readable source containing concrete information and many source references. They are producec by authors belonging to organisations or by guest authors (sometimes scientists) which have been invited. The publishing or paying organisations may be:
The level of the reports published by these organisations can be very high, and sometimes even scientific standards are used. In almost all cases, however, there are parties involved who commissioned the publication and who sometimes determine which outcomes are presented to the outside world and which are not.
Even if you do all the checks on these pages there are no guarantees that the pieces you will be reading are true, objective and unbiased. You will always have to remain careful and above all, read critically. Make a sport out of spotting the bias in articles, books and reports.
If you want to use information from websites other than regular scholarly articles and books you should be extra careful and think about the role you give that information in your argument or analysis.
Ask yourself the following questions when reading webpages and be extra careful if the answer is negative most of the times.
These questions do not relate of course to the use of scientific articles or e-books published on websites.
Whether you can use non-scholarly sources depends on:
You can use non-scholarly sources as:
There are many hundreds of scholarly publishers. In addition to the large ones you will find below, there are many small ones in all language regions and there are also many universities that have their own scientific publisher (usually called ... University Press)
Content
Use reviews to:
To be found in:
Search with:
Sometimes entire publications are imitated having as aim satire, criticism, identity damage or commercial gain. That we are dealing with a fake publication may be clear from the start, but it may be done so subtly that you hardly notice it. In that case the exposing of a spoof is a question of experience and reading and viewing very carefully
NRC Next became victim of a spoof in the build-up to the 2012 elections of the Dutch parliament