Methodology

The 30 months USAID Strengthening Media in Macedonia Project – Media Fact-Checking Service Component implemented by the Metamorphosis Foundation for Internet and Society (www.metamorphois.org.mk) aims to empower Macedonian citizens to hold the media accountable, and to assist journalists in the implementation of their professional standards, by providing online tools and resources for public education and awareness raising. The purpose of the Media Fact-Checking Service Component is to increase the citizen demand for fact-based, objective and professional news and information.

Daily article reviews

The daily articles are analyzed according to several key criteria according to which the research questions are posed, and the answers of which identify the different parameters that actually define the “assessment” of the reviewer of the analyzed article. The selected parameters are then textually elaborated, resulting in a final form of the review composed of easily perceivable (graphically represented) qualitative assessments and a text part which explains the reviewer’s selection of specific parameters. The selection of parameters does not necessarily reflect a negative or positive assessment. The work of the reviewer is to include the assessments of the analyzed media article in a clear text and thus define the perception about its quality.

The daily articles cover informative journalistic genres. Given that different classifications offer different definitions of the types of journalistic content and the fact that the boundary between some subtypes is very thin, the methodology does not impose a precise identification of the genre of the analyzed article. Ljubomir Rajnvan highlights two aspects in terms of genres: their application in practice and their treatment in theory. Namely, the first aspect is based on the experience of good journalists who creatively uphold and apply the purity of genre expression, whereas the theoretical aspect instigates a number of challenging issues and necessary precautions in terms of possible patterning against creative journalism. According to him, “Classification must not be treated as an absolute category for mechanical application, but rather as a flexible framework that needs to be used creatively[1]”. This approach is generally acceptable as a compromise between the theoretical and the practical.

However, the basic review criteria is for the selected article to contain a group of specifics that serve as identifying signs of the journalistic product as informational, starting from its content and form. According to this, any informative journalistic expression characterized by a value minimum ​​of the information and we know that it suits the personal interests and needs of the readers, refers to important individuals, groups of people or events, is topical and has a certain influence on the readers’ feelings, is suitable for this type of analysis. Reviews are not limited only to negative examples of journalistic practice; on the contrary, the methodology aims to incite reviews of positive examples of journalistic work, in accordance with the offer of the Macedonian online media space. In addition, the review selection is influenced by the impact of the online media.

Articles are not selected by scanning all the daily online articles and selecting the most influential, most negative or most positive examples. Typical cases are selected instead, in accordance with the aforementioned selection criteria, and are primarily suggested by the reviewers themselves, the readers, the editor or another team member.

During the day, members of the team of reviewers create a list of articles suitable for analysis, in agreement with the editorial team, and those articles are further distributed for analysis. This process uses mailing lists, Facebook groups and other informal methods of online and offline communication. The suggestions of readers are marked and publicly visible, and the process of reviewing will only include suggestions that match the criteria specified in this methodology.

1. Truthfulness of facts

 –          Presence of facts

–          Accuracy of facts

–          Reliability of facts

Explanation: A newspaper article that belongs to the group of informational genres should be based on facts in order to justify its basic purpose – informing the reader. Then those facts must be accurate and reliable. While the presence of facts and their reliability for a journalist in the role of a reviewer can be easily determined, the accuracy can be confirmed or denied through research work (availability of other resources) and by direct testimony about the event.

2 Information sources

–          Presence of information sources

–          Type of information sources

–          Relevance of information sources

–          Number of information sources

–          Pluralism (versatility) of information sources

–          Relevance of information sources

Explanation: Information sources should be checked, confirmed and attributed. A source is official when it is quoted by name and last name or as a specific institution or company; sources are unofficial when they are not quoted by their name and last name, but as an institution. Sources are anonymous when they are neither located nor quoted at the request of the source. Sometimes, in journalistic practice certain declarations such as: “from a close source”, “according to officially unconfirmed information,” are used, leaving room for maneuvers and creating a sense of doubt among the readers.

Information related to a current event and constantly topical issues are at the core of journalist work. Despite the journalists’ freedom to present topics that they consider to be important for citizens at a given moment, they are also responsible for checking the temporal and spatial framework of information that they plan to use as a basis for the announcement. Lately, there is a tendency of using past information and developments, as well as information pertaining to a particular geographical area and placing them in a current, local or national context. This not only misinforms readers, but can also be a powerful weapon for manipulation and can have a negative impact on public awareness and public opinion.

3. Extensiveness

 –          Comprehensiveness

–          Information concealment

Explanation: The basis of every journalistic genre, or article and report, is to convey a certain message, to have a purpose and impact on public opinion. In this way the journalistic product features one of the basic qualities of journalistic expression – extensiveness.

The journalist should try to make the article comprehensive, i.e. to contain more information and to present more experiential, official and expert sources, depending on their objective availability. Journalists are not to be held accountable for the unavailability of certain sources and information, or to be blamed for the deliberate refusal of sources to communicate with them. However, in the case of sensitive news, especially news containing information of accusatory character, the journalist is obliged to attribute such allegations, which is considered as a minimum level of comprehensiveness of the article.

On the other hand, the intentional or unintentional exemption of parts of the information constitutes information concealment, which can be used for misrepresentation of the truth, propaganda, and inflicting damage to a person, company or some other entity.

4. Partiality

 –        Presenting the truth

–        Twisting the truth

–        Lie

–        Balance in presenting sides

–        Types of bias

Explanation:  Media bias refers to the selection of the events covered, and the way in which they are covered. There are several types of bias, mostly political bias, i.e. unacceptable manipulation with the news by twisting and spinning segments of the article, segments that are only partially true in order to present them as positive or to inflict political damage to some political party, ideology or person.

The cases in which journalists and media advocate for and promote political views are acceptable if they clearly separate facts from opinion, and do not conduct the editorial policy in a manner that endangers the objectivity and balance of reporting. In addition, cases of commercial bias are also reviewed in order to determine the media and journalists’ affiliation with a particular company or brand.

5. Commenting

 –        Presence of commenting elements

–        Separated versus integrated commenting

 Explanation: In the absence of sufficient facts for writing an article or a report, journalists (especially young journalists) often reach for the comments. Even if he/she opted for this journalistic form, the journalist should keep a certain emotional distance from the developments and his/her views should be based on arguments. However, this situation in practice results with articles placing the commenting elements in the informative genres, or even worse, inserting those elements as an integral part of the facts. The difference in journalistic genres is in the way information is being presented; therefore only presented information is actually news.

Commenting should be clearly separated from the article in order to prevent it from burdening the article with the subjective discourse of the journalist or media.

6. Plagiarism[2]

 –        Partial or complete copy

–        Attribution of the source or author

 Explanation: Plagiarism as per this methodology is when a work is created by copyright infringement of a third (physical) person or a legal entity, by presenting some author’s complete or partial media contents; without the permission of the author or copyright holder, without specifying authorship data or using modified authorship data.

In journalism, plagiarism is a violation of journalistic ethics. Article 12 of the Code of Journalists of Macedonia: “Plagiarism is unacceptable. Citations may not be used without specifying the source or the author.”

Reviewers inspect whether a particular work is the product of plagiarism based on the following criteria:

1. The content seems to be a complete or partial copy
2. The author or the source is not attributed, or segments are conveyed in a different form instead of the original one
3. There is a direct complaint from the original author/publisher expressed publicly or directly to the Service in a form that can be documented
4. There is a violation of the terms laid down in any of the free Creative Commons licenses.

The reviewer can determine the time of publishing of an original article (date, hour and minute) based on publicly available information on the media’s website and can compare this data with the time of publication of the article suspected to be a copy.

7. Quality of the title

 –        Informative

–        Creative, witty

–        Sensational

–        Suitable

–        Clear

Explanation: An appropriate title is a title that is informative (when announcing the contents of the article), creative, witty, and even sensational, whereas an inappropriate title is one that misinforms and misleads the reader, with purposes that are unacceptable from an aspect of the basic standards of journalistic ethics.

8. Photograph

 –        Presence of a photograph

–        Suitability of the article photograph

–        Use of photographs for manipulation

–        Photograph authorship

Explanation: Publishing photos out of their original context can be a dangerous manipulation that can compromise people’s privacy or damage the reputation of companies and other entities.

9. Hate speech

 –        Offensive speech and discreditation

–        Hate speech

–        Inciting violence

–        Discrimination

–        Types of discrimination

Explanation: Journalistic articles should respect human rights and liberties, including the cultural, religious and ethnic differences, without using hate speech and without inciting violence and discrimination based on religion, nationality, race, sex or any other grounds.

In the absence of a definition in the Macedonian legal system, hate speech may be defined as a speech that promotes or encourages violent acts or offenses committed out of hate and a speech that creates grounds for hatred and prejudices, which in turn, may ultimately be the reason for such crimes.[3]

Several basic research questions are introduced here, but if the reviewer identifies different aspects of violation of basic human rights and freedoms, particularly in terms of hate speech, the list can be extended. In this stage of the review, the two methodologies are connected.

10. Optimized for the web

 –        Readability

–        Multimedia quality

–        Presence of relevant contextual links

–        Orderliness

–        Use of tags

Explanation: Internet content is characterized by a series of characteristics that stem from the nature of the media on which they are published. Newspapers highlight important article titles, place the text in frames and illustrate the elements of the page; television and radio highlight the presented information audibly and visually, so web media also needs to adapt the content intended for publishing on the web.

This means that they should primarily be readable (font size and type, contrast between the text and the background, dividing paragraphs, dividing large texts with titles in-between) and include photographs and video materials with a good quality, appropriately edited (appropriate size) and arranged in relation to the text, use contextual links whenever relevant, use stylistic editing capabilities (citations, different levels of titles, etc.) and to enable the display of related content, based on the use of appropriate tags.

Once the journalist reviewer determines the parameters in each of the ten criteria, the reviewer textually explains the selection for the parameters that affect the quality of the presented information and completes the review with an introductory and conclusion segment.

Critical discourse analysis

The second team comprised by analysts and experts in the field of human rights, produces two analyses a week covering a wide range of topics in the field of ethical journalism. The selection of topics is done based on predefined research questions, covering seven thematic areas.

The texts are written in an analytical style and use tools and methods of critical discourse analysis applied to the Macedonian media discourses. In order to determine the perspective or the point of view for presenting the social reality through the media, it is necessary to analyze the techniques used by media discourses to distinguish details (choice and position of photographs and other graphic content; use of headings, subheadings, keywords to be highlighted; techniques of exclusion and marginalization; use of selective votes to send the desired message…).

In addition to these techniques, critical media discourse analysis uses other techniques for revealing the media perspective, power relations, discrimination and existing strategies to strengthen (or put into question) the existing stereotypes and prejudices, insinuations, connotations, tone, use of first or third person, rhetorical, argumentative and interpretative strategies, etc.

So far, the techniques of critical discourse analysis are widely applied in both the analysis of media discourses, and the analysis of phenomena in the field of human rights (racism, xenophobia, sexism, homophobia, hate speech, negative stereotyping and profiling …) which makes them particularly suitable tools for analyzing media content, taking into account the existing standards of ethical journalism.

Below there is an explanation of the fields of analysis, the elements that belong to each area, as well as the research questions posed during the selection of media content for analysis. This list of areas, elements and questions remains open and can be further extended in accordance with the established trends in Macedonian online journalism.

1. Media, diversity and non-discrimination

The category will include negative stereotyping and prejudices, generalizations, sexism, homophobia, xenophobia, racism, transphobia, ageism, negative profiling, imposing cultural values, division to WE – THEM, sensationalism and ignoring the voice of marginalized people.

–          Cultural differences

–          Religious differences and differences in conviction

–          Gender identity and gender roles

–          Sexual differences

–          Political differences

–          Ethnic differences and racial differences

–          Belonging to a marginalized group

–          Media treatment of persons with disabilities

Research questions:

1)      Is there any negative stereotyping and reinforcement of prejudices (especially towards members of vulnerable groups)?

2)      Is there any generalization based on a single case?

3)      Is there any sexism, misogyny, imposed values of heteropatriarchy, or disregard for sex and gender differences and gender identities?

4)      Is there any homophobia, transphobia or disregard for sexual differences?

5)      Is there any xenophobia, hate speech and intolerance towards foreigners?

6)      Is there any negative profiling of members of certain groups?

7)      Is there any ageism, or disregard of differences based on age?

8)      Is there a violation of cultural differences and imposition of certain cultural values ​​over other cultural values?

9)      Are there any divisions such as WE – THEY?

10)  Is there a sensationalistic approach that overshadows discrimination or the disregard of differences?

11)  Are the members of marginalized groups disrespected and is the voice of marginalized people ignored?

12)  Is there a disregard for religious differences and differences in conviction, or an attempt to impose certain religious values?

13)  Is there a disregard for ethnic differences?

14)  Is there a disregard for racial and skin color differences?

15)  Is there a disregard for political differences or political affiliation?

16)  Is there a disregard for social, educational or linguistic differences?

17)  Is there a disregard for people with disabilities?

2. Law enforcement and human rights protection

–          Violation of the presumption of innocence

–          Disclosure of the identity of a suspect, a victim, a minor or protected witness

–          Victimization

–          Selective approach (not reporting about all parties involved in the dispute, sensationalism, uncritical presentation of official statements,

–          Suggesting a ruling

Research questions:

1)      Is there a violation of the right of presumption of innocence, when reporting on police and court cases?

2)      Are there any breaches of privacy (disclosure of the identity of the suspect/accused, the victim, a minor, a protected witness…) when reporting on police and court cases?

3)      Is there a selective approach (not reporting about all parties involved in the event or dispute) when reporting on police and court cases?

4)      Is there an uncritical presentation of official statements, favoring only the official story, when reporting on police and court cases?

5)      Is there an unethical sensationalist approach to reporting on police and court cases?

6)      Is there a suggestion for a decision, ruling or punishment when reporting on police and court cases?

7)      Is there any insensitivity or disregard for the right to a fair trial (and the principle of equality of arms) when reporting on court cases?

3. Privacy

–          Home

–          Sexuality

–          Marriage, family, parenting, household, communities, relationships

–          Identity

–          Communications

–          Unauthorized recording and display

Research questions:

1)      Is there any intrusion in the privacy of the home by the journalist or the media?

2)      Is there any intrusion in the sexual privacy, sexual orientation, sexual identity, informal community, intimate relationships, sexual relations, sexual relationships?

3)      Is there any intrusion in privacy related to marriage, family, parenting and household?

4)      Is there a violation of the right to a personal identity?

5)      Is there a privacy violation by unauthorized intrusion in communications?

6)      Is there an unauthorized recording violating privacy?

7)      Is there an unauthorized display of recordings violating privacy?

4. Freedom of expression versus hate speech

–          Courage to seek information

–          Courage to convey information

–          Courage to confront the editor/owner in case of a violation of the Code

–          Undistinguished right to freedom of expression

–          Typical cases of hate speech in accordance with the definitions of the Council of Europe

–          Inciting violence

–          Harassment and bullying

Research questions:

1)      Is there a lack of courage for seeking information?

2)      Is there a lack of courage for conveying information?

3)      Does a journalist lack courage to confront by the editor/owner in case of violation of the Code and professional standards?

4)      Is there insensitivity and ignorance when it comes to the right to freedom of expression?

5)      Is there hate speech?

6)      Is there any incitement to violence?

7)      Is there harassment and bullying?

5. Dignity, honor, reputation

–          Degradation and humiliation

–          Labeling

–          Insult and defamation

–          Discrediting and disqualification ad hominem

Research questions:

1)      Are there any demeaning or derogatory expressions?

2)      Is there any labeling?

3)      Are there any insults and defamation?

4)      Are there any discrediting attempts?

5)      Are there any attempts for ad hominem disqualification?

6. Critical thinking and debating

–          Challenging the authority

–          Critical questioning

–          Critical argumentation

–          Previous critical analysis and preparation

–          Encouraging a debate

Research questions:

1)      Are there any perceptible efforts for critical challenging of authority?

2)      Is there a recognizable sense for public interest?

3)      Is there a critical approach towards controversial topics?

4)      Are the information sources asked any critical questions?

5)      Is there a critical argumentation?

6)      Is there a previous critical analysis and preparation for the topic?

7)      Is the text encouraging a critical discussion and debate on a topic of public interest?

7. Manipulations and propaganda

–          Abusing a media for confronting individuals

–          Lack of professional distance from political parties

–          Concealment of essential data

–          One sided data emphasis

–          Deliberate distortion of facts

–          Presenting perceptions and views as facts

–          Forgery and document tampering

–          Fabricating data and public opinion (surveys, etc.).

–          Manipulations in argumentation

–          Propaganda techniques

Research questions:

1)      Is the media abused for confronting individuals and thereby violating human rights?

2)      Is there a noticeable lack of professional distance from political parties?

3)      Is there a cover-up of essential data, with the purpose of manipulating the public?

4)      Is there a one-sided data emphasis in order to manipulate the public?

5)      Is there a deliberate distortion of facts in order to manipulate the public?

6)      Is there a manipulative presentation of perceptions and views as facts?

7)      Are there any attempts for forgery and documents tampering?

8)      Is there fabrication of data and public opinion (surveys, etc.)?

9)      Are there any noticeable manipulations in the argumentation?

10)  Are there any other noticeable propaganda techniques?
Educational articles

The so-called journalistic lessons are dealing with journalistic standards from a broader aspect and are not limited to an analysis of a single article by the media, but focus on analyzing a certain phenomenon, trend and educational elaboration of the subject for the purpose of teaching by using positive examples and providing guidelines for enhancing the quality of work. These articles are inspired by the journalistic codes for ethics, professional journalistic literature, the literature devoted to online publishing, and from the practice in Macedonia.

Criteria for selection of journalists-reviewers

Journalists-reviewers who are members of the team have the following tasks:

• identify important daily news;
• conduct reviews of such content;
• write longer articles for public education on the implementation of journalistic standards.

Candidates for team members should meet the following criteria:

• completed studies in the field of journalism or a related field;
• at least a 2 year experience in the field of media;
• analytical skills;
• good command of written Macedonian or Albanian literary language;
• their professional engagement in media should be based on the Code of Journalists of Macedonia or any equivalent code of ethical and professional standards adopted by a relevant organization of journalists (this commitment for acceptance of the self-regulation standards should be explained in the cover letter).

Priority will be given to candidates who:

• are not currently employed in Macedonian media;
• members of professional associations of journalists who are committed to promoting professional standards;
• are able to professionally monitor and review media in Macedonian or Albanian.

Candidates should submit their CV, cover letter and links to three of their articles or scanned posts. Shortlisted candidates will be interviewed, and the most appropriate candidates will be selected based on their qualities, experiences, availability and interests, and depending on the current need to expand the team.
Criteria for selection of analysts

Analysts who are members of the team have the following tasks:

• identify cases of violation of professional journalistic standards in the media, including violations of human rights or trends in the quality of the media in a broader context;
• conduct analyses based on critical discourse analysis for such cases.

Candidates for members of the team should meet the following criteria:

• to be knowledgeable in the field of human rights (professional engagement, education, NGO activities);
• experience in writing research reports, columns and/or analyses
• are not/have not been political party officials;
• good command of written Macedonian or Albanian literary language;
• their professional engagement in media should be based on the Code of Journalists of Macedonia or any equivalent code of ethical and professional standards adopted by a relevant organization of journalists (this commitment for acceptance of the self-regulation standards should be explained in the cover letter).

Priority will be given to candidates who:

• have already participated in media monitoring
• are members of human rights associations
• are not currently employed in Macedonian media;
• are able to professionally review media in Macedonian or Albanian.

Candidates should submit their CV, cover letter and links to three of their articles or scanned posts. Shortlisted candidates will be interviewed, and the most appropriate candidates will be selected based on their qualities, experiences, availability and interests, and depending on the current need to expand the team.

About the methodology

This methodology was created as a result of the analysis of the context in Macedonian online journalism, consultations with experts, professional journalistic literature, tests and successful examples from the world.