A Summary Of Interrater Reliability

Dunbar, G. (2005). Evaluating Research Methods in Psychology. New Jersey: John Wiley & Sons Inc.

Correspondence concerning this article should be addressed to Christina McElroy, Department of Psychology, Introduction to Research, Liberty University, 1971 Liberty University Drive, Lynchburg, VA 24515. E-mail: cmcelroy1@liberty.edu

The sample for this study consisted of 222 participants who were second year psychology students from the University of Newcastle. All students were participating as part of a course requirement and all had given their consent to participating in the study.

Inter-rater reliability is to what extent do two or more informants agree with what they have observed. It is extremely important when doing researches since it can weaken the accuracy of the result. If there are a lot of disagreement between different informants, the findings then will not be considered as the correct representation of the variables measured. Inter-rater reliability is also essential when clinicians are making a diagnosis. There could be some unavoidable biases from the clinicians while interviewing the patients; thereby, there might be false interpretation towards the severity of symptoms.

In order to know whether the evidence of research studies are accurate, one must be able to have a fundamental understanding in statistical analyses to determine if such descriptions and findings within manuscripts and articles are presented correctly and explicitly (Sullivan, 2012). Proper use of statistics begins with the understanding of both descriptive and inferential statistics. Correct organization and description of data characteristics from the population sample being studied leads the researcher to identify a hypothesis and formulate inferences about such characteristics. It is with inferential statistics that researchers conduct appropriate tests of significance and determine whether to accept or reject the identified null

There has been an ongoing debate on whether psychology is indeed scientific, although recent, psychology is now considered a science. This is because it uses scientific methodology in researching, devising treatments and measuring the outcomes. These methods include collecting and analysing data and concluding their findings in order to identify whether the research or treatment adequality solves the problem. Scientific studies must be replicable, this means the if repeated exactly the same, the results should produce an identical outcome. Replicability can be increased by ruling out any alternatives that may not have originally been thought of. Objectivity in science is the idea that scientist, in an attempt to get the best results, must first

In the text book, “Theories and Research of Personality” written by Daniel Cervone and Lawrence A. Pervin, the authors talk about the goals to research and they are referring to reliability, validity, and ethical behavior. With reliability, the author is referring to the “extent to which observations can be replicated and whether the measures of the research are dependable or stable” (Cervone, Pervin 43, 2013). Reliability is extremely important to have when conducting research because if the research conducted is not reliable then when trying to get research out to people, other psychologists will not believe what you are trying to get across and in the long term affecting ones career. Also Cervone and Pervin talk about validity which is,

For any measure to be valuable in psychological research, it needs to be both valid and reliable (Goodwin, 2008: 128). Research is reliable when more researchers have found the same results, or, within for instance behavioural research, when the same behaviour occurs at several measurements (Goodwin, 2008: 124). There are different types of validity. Firstly, there is construct validity, which measures whether an operationalisation of a construct actually measures what it is supposed to measure. Secondly, criterion validity determines whether a certain phenomenon is related to another phenomenon, and can accurately determine future developments. Lastly, content validity determines whether a test measures all aspects of the construct that is being measured (Goodwin, 2008: 125-126).

Within the eight weeks of this course, we learned a lot about psychological research. This course provided an understanding of how research is completed. This course also gave an understanding of what methods to utilize based on that particular research that is being conducted. In this paper I will discuss exactly what I learned about research at an introductory level as a case study. In the field of mental health, professionals must be constantly looking for ways to help their patients (Cozby & Bates 2015).

There is no evidence about BIMOR reliability and validity measurement. However, for the purpose of this study reliability and validity measures will be performed. In order to present evidence of reliability a pilot study will be developed. In this pilot study 10 freshman male college students will be interviewed using BIMOR. These students’ responses will not be part of the final study. Once interviews are held, the researcher will perform an Interrater reliability method, in which two experts in applying BIMOR will interview 5 students each. After gathering the students’ responses to the interview, the researcher will calculate the Interclass Correlation coefficient of each of the items in BIMOR. The validity of BIMOR will be evidenced by

A research is reliable when the measurement instrument can generate the same outcomes consistently. This section will discuss how the researchers have endeavoured to maximise reliability by minimising method error and maximising the inter-rater reliability and internal consistency of the research. Suggestions on how to further increase the study’s reliability are also given.

Reliability refers to coherence, stability and dependability in test results, generally using internal consistency to express the levels of reliability in the test. The higher reliability indicates the higher level of accordance, stabilization and dependability in test results. Reliability is the precondition of validity (Guba and Lincoln, 1981). The same findings may not generate if the same research is repeated, because many influencing factors may work in the process of research. The process of establishment in reliability research includes: the research rigorously collect and explain data in consistent investigation (internal checks); the process is transparent (sample design, field work, inquiry and rational data). Patton (1987) suggests that the use of triangulation in multiple approaches can increase the reliability in results.

At the beginning of this subterm, I had a limited understanding of the proper research methods used within psychology. Despite having previously completed several psychology courses, two being upper level courses, there was still much I needed to learn pertaining to psychology research methods. While this research class has been on only an introductory

Reliability is defined, within psychometric testing, as the stability of a research study or measure(s). Reliability can be examined externally, Inter-rater and Test-Retest, as well as internally; which is seen in internal consistency reliability methods.

Empirical research and data are at the heart of psychological science; although other qualitative methods are sometimes used, it is quantitative research methods that underpin psychology as a legitimate and reliable science. As an empirical science, psychology has a well-entrenched tradition of Null Hypothesis Significance Testing (NHST) (Fritz, Scherndl, & Kühberger, 2012). Despite its limitations, and extensive criticism, NHST remains the most commonly used method of statistical testing in psychology (Bakker & Wicherts, 2011; Fritz et al., 2012). In an attempt to create a consistent, replicable, and cumulative body of psychological research, official reporting guidelines, encompassing the major components of research papers (from method