Wheeler, Richter, & Sahadevan asked the ten authors of their "great" journal articles why they felt their work were so well received. Analyzing the compiled interview data, they found researchers credit four central factors to their success: "1) timing of the article or addressing a need in the literature; 2) defining or clarifying constructs in the article; 3) focusing on multiple levels of the organization; and 4) becoming a foundation article" (p. 1328). Wheeler, Richter, & Sahadevan’s criteria allows one to move beyond the citation counts of articles and impact factors of journals to examine the qualities of exemplary research, particularly within the domain of organizational behaviour. Applying this framework to the Latham, Erez, & Locke article assists in illuminating the work’s key strengths that make it an exemplary work.
Background
Latham, Erez, & Locke’s article, entitled "Resolving scientific disputes by the joint design of crucial experiments by the antagonists: Application to the Erez-Latham disputes regarding participation in goal setting", was originally published in the prestigious Journal of Applied Psychology in 1988 and then included in the edited volume Doing Exemplary Research by Frost & Stablein in 1992. The two lead authors, Gary Latham and Miriam Erez, had in their prior work on participation in decision making (PDM) received contradictory findings. Erez’ and Latham’s mutual acquaintanceship with Locke lead to the three meeting and brainstorming possible explanations for their research discrepancies. Rather than follow the seeming tradition of slinging scholastic mud at each other, Erez and Latham decided to collaborate on a joint design of experiments to uncover their prior research differences. Locke acted as mediator to ensure a smooth process, although he states that he rarely had to intervene and never in a significant manner (Latham et al, 1988). They conducted a series of four iterative experiments, using their learnings from early experiments to guide subsequent experiment design. Their process of painstakingly analyzing their research design enabled them to uncover procedural differences and conflating variables that accounted for their prior oppositional findings. Their results constitute a crucial experiment that proved that PDM does not increase performance compared to supportively-stated goals, but that it does increase goal commitment. Through this research they made a strong addition to the knowledge on PDM and pioneered a method for collaborative problem solving.
Non-exemplary qualities
No research is ever entirely great. The authors interviewed by Wheeler, Richter, & Sahadevan all commented that they believed their work could be improved. Omissions, compromises, or errors are made for the sake of expediency or arising from a lack of omniscient knowledge. Before extolling a work’s virtues, I feel it is important to acknowledge where it falls short. Not wishing to fault research for the assumptions and traditions of its epistemological paradigm, I do believe Latham, Erez, & Locke’s article lacked clarity and evocativeness in the writing style. A complaint that experimental laboratory tests lack ecological validity is not original or entirely fair as findings from experiments have been shown to transfer outside of the lab. Nonetheless, decisions in the test task chosen, performance measurement rubrics, and test duration limit the study’s ability to be generalized to larger, field scenarios. The study is further limited in its generalizability by the authors’ sampling strategy. Despite the ubiquity in academia of drawing upon a convenient supply of university students for research (even giving them course credit, as was the case with this study), sampling from university students, and specifically business students, does limit a study’s population validity. University students, and in particular business majors, are not representative of the general population as they may possess non-standard levels of intelligence, ambition, and literacy. Finally, I believe the study failed to control for, or at least acknowledge, possible conflating variables. The study makes definitive claims about the success of participatory decision making, yet failed to examine or control variables such as personality types, gender, age, power structures, stress level, and worker morale. In acknowledgement of Erez' Israeli background, they do posit that cultural factors may play a role, but this is the extent of their controls for the possible role of social, psychological, or demographic factors. Accounting for all possible mitigating factors is infeasible in any research; yet failing to concede their possible role weakens the authors' claims. These problems, however, are not unique to this study and do not detract from the strengths of the work.
1) Timing of the article or addressing a need in the literature
The first criterion for quality research is the degree of its timeliness or filling a void in existing literature. It is the latter point, that I personally believe touches upon the work’s most exemplary quality: its innovative approach. Cummings and Earley believe this approach is novel and effective; they recommend it be adopted by other researchers as it leads to a full understanding of phenomena (1992). In the history of intellectual thought and research, it is likely that scholars with opposing views have collaborated before. Within academic literature "antagonist" collaboration appears rare enough that Latham, Erez, & Locke believe they have pioneered it (1988). Although this claim is not substantiated, it appears no one has refuted their claim. Latham, Erez, & Locke may not entirely have invented collaborative and mediated research, but they are clearly pioneers in its refinement and promotion.
I admire their approach as it offers a constructive alternative to academics with divergent views from arguing and belittling each other in protracted arguments in literature and at conferences. It appears Greenberg is similarly impressed by this innovative approach noting, "Instead of merely adding to derision, as so often occurs, the authors agreed to some crucial experiments designed to answer a key question to their mutual satisfaction. We need more of this kind of work, in my opinion" (as cited in Wheeler, Richter, & Sahadevan, 2004, p. 1337). Greenberg and Cummings & Earley, and Greenberg all comment on the rarity of any such collaboration, which may be due to the risks involved. Either researcher could have been exposed as incorrect or having conducted prior faulty research. I find this an exemplary quality of the researchers that to me reveals their commitment to quality scholarship over protecting their own ego or reputations. I thus commend Erez and Latham both for pioneering this innovative approach and for committing to it.
2) Defining or clarifying constructs in the article
A key problem with Erez and Latham’s earlier work, and a problem not unique to them, is that there were unstated assumptions, unknown conflating variables, and undocumented procedures. Another additional exemplary trait of Latham, Erez, & Locke’s article is their how their collaboration enabled them to uncover these mysteries to deliver a solid, rich understanding of the phenomenon. This quality is remarkable as Wheeler, Richter, & Sahadevan state, "articles that become ‘great works’ define or clarify constructs in a way that future researchers rely on" (2004, p. 1329). Frost and Stablein in their commentary of their exemplary works believe that researchers' attention to detail (p. 249) leads to quality research. Precision in research design is crucial not only to ensure a study’s reliability and internal validity, but more importantly to deliver construct validity. It seems evident from Erez and Latham’s writings and commentary on the article that their previous studies lacked construct validity – they were not studying exactly what they thought they were studying. Their studies had conflating variables that meant that prior control groups were exposed to differing treatments. In addition, their operational definitions of a key construct, what constitutes a group, differed fundamentally. Erez defined it as a group of five people with equivalent power, whereas Latham defined it as a dyad with a superior and subordinate. Such differences not only limit the ability to compare findings across studies, but undermine the validity of the study. Latham appears to have been studying a completely different phenomenon than what he states as his definition is at odds with common definitions of a group and fails to account at all for the overriding power dynamics of his dyads. Instead of studying PDM, Latham was studying boss-employee dynamics. Erez describes how such differences and omissions can arise:
Very often, researchers are even unaware of contextual differences because they are part of the context and have no external reference point. By working together, the two researchers provided each other with the reference points needed to define the unique characteristics of each other’s procedures…. The most striking part of this process for Locke was the number of differences in procedures and design that can occur when two people are allegedly studying the same phenomenon. The number of little differences between studies can add up and have a significant effect on the differences in empirical findings and hence on conclusions (1992, p. 162).It is crucial to clearly operationalize the constructs being studied and control for (or account for) as many variables as possible, to be able to make definitive claims about a phenomenon. Campbell, in commenting on the article, commends the authors for their clear, unambiguous operationalizing of key constructs. He observes it is more common for researchers to "underspecify the nature of the variables we use" which creates "surplus meaning" (1992, p. 174). Citing the Hawthorne studies, Campbell believes a lack of clarity on what exact variables were manipulated leads to speculation over the results. Latham, Erez, & Locke may not have accounted for all possible variables, but their efforts to uncover their differences resulted in a study with a high degree of construct validity that allowed causal relationships to be determined about a precise phenomenon.
3) Focusing on multiple levels of the organization
Wheeler, Richter, & Sahadevan’s third quality criterion is less well articulated than their other criteria. They offer little explanation beyond that quality work should "interest both organizational scholars and practitioners" (p. 1329). One can infer from their cited examples that research should appeal beyond narrow, cloistered corners. It is not surprising that scholars writing on business and management topics would value a study’s application beyond academia, considering the subject’s direct relationship with industry. The subject of Latham, Erez, and Locke's article is organizational behaviour, which is generally more readily applicable to field contexts than other academic subjects. More specifically, Latham, Erez, & Locke do present a phenomenon with implications for both scholars and practitioners. Their work centers on a common organizational activity, goal setting, and its outcomes, performance and commitment. Practitioners can apply the findings to their own work, while scholars are offered an examination that addresses both praxis and ideology. By focusing on a research problem with direct application to business settings, as well as addressing the larger issue of power and equality, the study is exemplary in its appeal to both academia and industry.
4) Becoming a foundation article
Regarding Wheeler, Richter, & Sahadevan’s final criterion, "becoming a foundation article", the authors believe that the degree of success in the preceding criteria will beget this one. Although Latham, Erez, and Locke have distinguished academic careers, it is arguable whether or not the article’s 110 citations in SSCI would designate this as a foundational piece. Examining the article in Google, however, does reveal that the article’s importance extends beyond refereed journals as it is referenced 262 times. The influence this article has for inspiring or enlightening scholars is less hard to pinpoint when it does not result in a citation. Initially, the article’s quality was affirmed by peers when the Academy of Management awarded it the outstanding publication in organizational behavior in 1988. More recently, in 2004 Professor Greenberg, one of Wheeler, Richter, & Sahadevan canonical authors, credits Latham, Erez, and Locke’s article as the work he most admires. The inclusion of the work in Frost and Stablein’s collection further adds to this work’s influence beyond its citation counts, as the book is reported to be required reading in many graduate classes (Rafaeli, 2009). Its continued relevance is also demonstrated by its citation history, provided by Scopus. Scopus credits the article with 76 citations since 1996, the year Scopus’ database begins. Since that year, however, the article has been cited every year - often up to eight or nine times. Being cited for over 20 years and the article’s more intangible influence as demonstrated by its textbook status does make a strong case that this work can be considered foundational.
Conclusion
The collaborative research by Latham, Erez, & Locke provides a model of a productive way to "add to our knowledge more efficiently and more rapidly than would otherwise be the case" (Latham, Erez, & Locke, 1988, p. 771). Their innovative approach resulted in a study which generated more insight into the phenomenon of PDM than their prior, individual work had uncovered. I believe they were able to generate this insight as their joint, even antagonistic, efforts lead them, even forced them, to conduct research with an exemplary degree of validity and reliability. Over twenty years after being published, Latham, Erez, & Locke’s article is still read and cited by scholars and practitioners thus maintaining its influence and claim as an exemplary work.
References
Campbell, J. P. (1992). Experiments as reforms. In P. J Frost & R. E. Stablein (Eds.), Doing Exemplary Research (pp. 173-176). Newbury Park, CA: Sage Publications
Cummings, L.L., & Earley, P. C. (1992). Comments on the Latham/ Erez/ Locke study. In P. J Frost & R. E. Stablein (Eds.), Doing Exemplary Research (pp. 167-172). Newbury Park, CA: Sage Publications.
Erez, M. (1992). Reflections on the Latham/ Erez/ Locke study. In P. J Frost & R. E. Stablein (Eds.), Doing Exemplary Research (pp. 155-166). Newbury Park, CA: Sage Publications.
Frost, P. J., & Stablein, R. E. (1992). Doing exemplary research. Newbury Park, CA: Sage Publications.
Latham, G. P., Erez, M., & Locke, E. (1988). Resolving scientific disputes by the joint design of crucial experiments by the antagonists: Application to the Erez–Latham dispute regarding participation in goal setting. Journal of Applied Psychology, 73(4), 753-772.
Rafaeli, A. (2009, March 11). Professor Anat Rafaeli. Technion - Israel Institute of Technology Retrieved February 10, 2010, from http://iew3.technion.ac.il/Home/Users/anatr.html
Wheeler, A. R., Richter, E., & Sahadevan, S. (2004). Looking back on their “great works”: Insights from the authors of great works in organizational behavior and human resource management. Management Decision, 42(10), 1326-1342.
No comments:
Post a Comment