Principles of Readability

Posted on March 30, 2013 by

4


DuBay, William H. “The principles of readability.” Impact Information (2004): 1-76.
Key Passages Findings: 
  • What is readability? Readability is what makes some texts easier to read than others. It is often confused with legibility, which concerns typeface and layout.  George Klare (1963) defines readability as “the ease of understanding or comprehension due to the style of writing.” This definition focuses on writing style as separate from issues such as content, coherence, and organization. In a similar manner, Gretchen Hargis and her colleagues at IBM (1998) state that readability, the “ease of reading words and sentences,” is an attribute of clarity.
  • The creator of the SMOG readability formula G. Harry McLaughlin (1969) defines readability as: “the degree to which a given class of people find certain reading matter compelling and comprehensible.” This definition stresses the interaction between the text and a class of readers of known characteristics such as reading skill, prior knowledge, and motivation.
  • Edgar Dale and Jeanne Chall’s (1949) definition may be the most comprehensive: “The sum total (including all the interactions) of all those elements within a given piece of printed material that affect the success a group of readers have with it. The success is the extent to which they understand it, read it at an optimal speed, and find it interesting.” (p.3)
Beginning early in the last century in the U.S., studies of the reading ability of adults and the readability of texts developed in tandem. Our subject matter falls under these headings:
  1. The Adult Literacy Studies:  These studies discovered great differences in the reading skills of adults in the U.S. and their implications for society.
  2. The Classic Readability Studies:  This section looks at the early readability studies, which started in the late 19th century and concluded in the 1940s, with the publication of the popular Flesch and Dale-Chall formulas. During this period, publishers, educators, and teachers were concerned with finding practical methods to match texts to the skills of readers, both students and adults.
  3. The New Readability Studies:  Beginning in the 1950s, new developments transformed the study of readability, including a new test of reading comprehension and the contributions of linguistics and cognitive psychology. Researchers explored how the reader’s interest, motivation, and prior knowledge affect readability. These studies in turn stimulated the creation of new and more accurate formulas. (p-3)
The major findings of the military research were:

  1. Measures of literacy correlate closely with measures of intelligence and aptitude.
  2. Measures of literacy correlate closely with the breadth of one’s knowledge.
  3. Measures of literacy correlate closely to job performance. Hundreds of military studies found no gap between literacy and job performance.
  4. Workplace literacy programs are highly effective in producing, in a brief period, significant improvements in job-related reading.
  5. Advanced readers have vast bodies of knowledge and perform well across a large set of domains of knowledge. Poor readers perform poorly across these domains of knowledge. This means that, if programs of adult literacy are to move students to high levels of literacy, they must help them explore and learn across a wide range of knowledge (Sticht and Armstrong 1994, pp. 37-38). (4-5)
  • U.S. civilian literacy surveys University of Chicago Study Guy Buswell (1937) of the University of Chicago surveyed 1,000 adults in Chicago with different levels of education. He measured skills in reading materials such as food ads, telephone directories, and movie ads. He also used more traditional tests of comprehension of paragraphsand vocabulary.  Buswell found that reading skills and practices increase as years of education increase.  He suggested that an important role of education is to guide readers to read more, and that reading more leads to greater reading skill. In turn, this may lead one to continue more education, thus leading to greater reading skill.  The National Assessment of Educational Progress (NAEP) of 1970-1971.  This study tested how students 9, 13, and 17 years old as well adults 26 to 35 years old perform on 21 different tasks. The results showed for the first time how age affects performance on the same items. This survey showed as children grow up, attend school, and become adults, they grow progressively more literate (Sticht and Armstrong, pp. 51-58).
  • Louis Harris survey of 1970:  The Louis Harris polling organization surveyed adults representing a cross section of the U.S. population. The subjects filled out five common application forms, including an application for a driver’s license and a Medicaid application.  The poll was the first of many to show that many U.S. citizens have difficulty with filling out forms. The Medicaid form was difficult, with only 54 percent of those with an 8th grade education or less getting 90-100 percent correct. Even many college-educated adults had trouble completing the Medicaid form (Sticht and Armstrong, pp. 59-62)
  • Adult Functional Reading Study of 1973: This study used household interviews to find out the literacy practices of adults. It used a second household sample to assess literacy skills. Over all 170 items used in the study, over 70 percent of the respondents scored 70 percent correct or better. As a trend, adults with more education performed better on the test than those with less. As with Buswell’s study, both literacy skills and literacy practices correlated closely with education. Book and magazine reading correlated more closely with years of education than did newspaper reading. Altogether, the adults reported that they spent about 90 minutes a day in reading materials such as forms, labels, signs, bills, and mail. (Sticht and Armstrong, pp. 63-66).
  • Adult Performance Level Study of 1971: This study began as a project funded by the U. S. Office of Education. It introduced “competency-based” education, directing adult education to focus on achieving measurable outcomes. By 1977, two-thirds of the states had set up some form of “competency-based” adult basic education. The test included over 40 common and practical tasks, such as filling out a check, reading the want ads, addressing an envelope, comparing advertised products, filling out items on a 1040 tax form, reading a tax table, and filling out a Social Security application. Results showed the high correlation between performance on all tasks and literacy (Sticht and Armstrong, pp. 67-98).
Young Adult Literacy Survey of 1985: This study of young adults (17-25) and the adult study that followed in 1992 both measured the literacy the same way in three areas:

  1. Prose literacy—meaning of selected texts
  2. Document literacy—finding information on a form such as a bus schedule.
  3. Quantitive literacy—mathematical and spatial tasks:  Both studies used a literacy scoring range of 1 to 500 and the five levels of skill defined by the National Assessment of Educational Progress (1985). John Carroll (1987) estimated the corresponding reading-grade levels as shown in Table 1. (p.6-7)
  • The National Adult Literacy Survey (NALS) of 1992: This U.S. Government study sampled 26,000 adults, representing 191 million adults. In 1993, it published the first of a number of reports on this survey entitled, “Adult Literacy in America” (National Center for Education Statistics 1993, 1999, 2001).  This study used the same tests as the Young Adult Literacy Survey and reported data with the same five levels of skill.  The data in this table suggest 40 to 44 million adults in the U.S. are in Level 1, defined as “functionally illiterate, not having enough reading skills for daily life.” Some 50 million are in Level 2. This means the percentage of adults who struggle at Levels 1 and 2 (below the 5th -grade level) in the U.S. reaches 48 percent.  The report confirmed that numeracy (quantitative) skills increase with reading skills. Adults of different reading skills not only have different worldviews but also different life experiences. Forty-three percent of adults with low-literacy skills live in poverty, 17% receive food stamps, and 70% have no job or part- time job. Over 60 % of frontline workers producing goods have difficulty applying information from a text to a task. More than 20% of adults read below the sixth-grade level, far below the level needed to earn a living wage. (p.8)

Challenges for technical communicators: The lessons of the literacy studies for technical communicators are obvious:

  • Low and intermediate literacy skills are a big problem for large numbers of users of technical documents. Providing technical documents at their levels will advance both their technical and reading skills.
  • The larger the audience, the more it will include the average reading habits and skills of the public as determined by the literacy surveys.
  • The more critical the information is for safety and health, the greater is the need for increased readability. The finding that the great majority of adult readers are mid-range, intermediate readers brings to us in technical communication new opportunities and challenges.
  • Intermediate readers represent a large audience that technical documents have been missing. Go into any library or bookstore, and you will find few technical or scientific publications in the “Young Adult” section, or elsewhere written at the 7th to 9th-grade level. On the Internet, there is the same scarcity of
  • intermediate technical materials.
  • For example, a small sampling of the author’s shows that the support sections of the Apple and Microsoft Web sites are written at advanced level of 10th grade and up. The technical books for Dummies and Idiots, while written in a casual style, are often at the 10th -grade level and up. Like the car-safety seat instructions, these technical documents are too difficult for 80 percent of adult readers in the U.S. Ironically, the user manual that comes with the CorelDraw program is written at the 7th-grade level, making it fit for a much larger audience than its Dummies counterpart.
  • Considering the keen interest that intermediate readers of all ages can have in technical matters, this literacy gap is troubling. While some highly motivated readers are able to master difficult technical materials, we cannot assume that everyone will do so. To the contrary, the difficulty of technical materials has taught many if not most readers of intermediate skill not to look for technical help in written texts. Helpful text means not only providing readers accurate information but also information written at the reading levels they need.
Ralph Ojemann: The difficulty of adult materials:  The year 1934 marked the beginning of more rigorous standards for the formulas. Ralph Ojemann (1934) did not invent a formula, but he did invent a method of assessing the difficulty of materials for adult parent-education materials. His criterion was 16 passages of about 500 words taken from magazines. He was the first to use adults to establish the difficulty of his criterion. He assigned each passage the grade level of adult readers who were able to answer at least one-half of the multiple-choice questions about the passage. Ojemann was then able to correlate six factors of vocabulary difficulty and eight factors of composition and sentence structure with the difficulty of the criterion passages. He found that the best vocabulary factor was the difficulty of words as stated in the Thorndike word list.  Even more important was the emphasis that Ojemann put on the qualitative factors such as abstractness. He recommended using his 16 passages for comparing and judging the difficulty of other texts, a method that is now known as scaling (See “Text leveling” below). Although he was not able to express the qualitative variables in numeric terms, he succeeded in proving they could not be ignored.
Dale and Tyler: Adults of limited reading ability After working with Waples, Ralph Tyler became interested in adults of limited reading ability. He joined with Edgar Dale to publish (1934) their own readability formula and the first study on adult readability formulas. The specific contribution of this study was the use of materials specifically designed for adults of limited reading ability. Their criterion for developing the formula was 74 selections on personal health taken from magazines, newspapers, textbooks, and adaptations from children’s health textbooks. They determined the difficulty of the passages with multiple-choice questions based on the texts given to adults of limited reading ability. From the 29 factors that had been found significant for children’s comprehension, they found ten that were significant for adults. They found that three of these factors correlated so highly with the other factors that they alone gave almost the same prediction as the combined ten. They were:

  1. Number of different technical words.
  2. Number of different hard non-technical words.
  3. Number of indeterminate clauses.
They combined these three factors into a formula to predict the proportion of adult readers of limited reading ability who would be able to understand the material. The formula correlated .511 with difficulty as measured by multiple-choice reading tests based on the 74 criterion selections. The Ojemann and Dale-Tyler studies mark the beginning of work on adult formulas that would continue unabated until the present time. (p.15)Lyman Bryson: Books for the average reader: During the depression of the 1930’s, the government in the U.S. put enormous resources into adult education. Bryson Lyman first became interested in non-fiction materials written for the average adult reader while serving as a leader in adult-education meetings in New York City. What he found was that what kept people from reading more was not lack of intelligence, but the lack of reading skills, a direct result of limited schooling. He also found out there is a tendency to judge adults by the education their children receive and to assume the great bulk of people have been through high school. At that time, 40 to 50 million people had a 7th to 9th grade education and reading ability.

Writers had assumed that readers had an equal education to their own or at least an equal reading ability. Highly educated people failed to realize just how much easier it is for them to read than it is for an average person. They found it difficult to recognize difficult writing because they read so well themselves.  Although college and business courses had long promoted ideas expressed in a direct and lucid style, Bryson found that simple and clear language was rare. He said such language results from “a discipline and artistry which few people who have ideas will take the trouble to achieve… If simple writing were easy, many of our problems would have been solved long ago” (Klare and Buck, p. 58).
Bryson helped set up the Readability Laboratory of the Columbia University Teachers College with Charles Beard and M. A. Cartwright. The purpose of the laboratory was not to rewrite the classics or to help the beginning reader. The purpose was to produce readable books on serious subjects for the average citizen.
Bryson understood that people with enough motivation and time could read difficult material and improve their reading ability. Experience, however, showed him that most people do not do that.
Perhaps Bryson’s greatest contribution was the influence he had on his two students, Irving Lorge and Rudolf Flesch. (p.16)Gray and Leary: what makes a book readable: William S. Gray and Bernice Leary (1935) published a landmark work in reading research, What Makes a Book Readable. Like Dale and Tyler’s work, it attempted to discover what makes a book readable for adults of limited reading ability.
Their criterion included 48 selections of about 100 words each, half of them fiction, taken from the books, magazines, and newspapers most widely read by adults. They established the difficulty of these selections by a reading-comprehension test given to about 800 adults designed to test their ability to get
the main idea of the passage.

No subsequent work has examined readability so thoroughly or investigated so many style elements or the relationships between them. The authors first identified 228 elements that affect readability and grouped them under these four headings:
1. Content
2. Style
3. Format
4. Features of Organization
The authors found that content, with a slight margin over style, was most important. Third in importance was format, and almost equal to it, “features of organization,” referring to the chapters, sections, headings, and paragraphs that show the organization of ideas (See Figure 4). (p.16-17)The New Readability studies were characterized by these features:

  • A community of scholars.  The periodical summaries of the progress of readability research (Klare 1952, 1963, 1974-75, 1984, Chall 1958, and Chall and Dale 1995) revealed a community of scholars. They were interested in how and why the formulas work, how to improve them, and what they tell us not only about reading, but also about writing.
  • The cloze test.  The introduction of the cloze test by Wilson Taylor in 1953 opened the way for investigators to test the properties of texts and readers with more accuracy and detail.
  • Reading ability, prior knowledge, interest, and motivation. A number of studies looked at the manner in which these reader variables affect readability.
  • Reading efficiency.  While other studies looked at the effects of readability on comprehension, these studies looked at the effects on reading speed and persistence.
  • The measurement of content. The influence of cognitive psychology and linguistics in the 1980s stimulated renewed studies of cognitive and structural factors in the text and how they can be used to predict readability.
  • Text leveling. Cognitive and linguistic theory revived interest in the qualitative and subjective assessment of readability. With training, leveling can be effective in assessing the elements of texts not addressed by the formulas.
  • Producing and transforming text. Several studies examined the effectiveness of using the formula variables to write and revise texts. When writers attend to content, organization, and coherence, using the readability variables can be effective in producing and transforming a text to a required reading level.
  • New readability formulas. Extensive studies of readability by John Bormuth and others looked at the reliability of a wide range of measurable text variables. They produced an empirical basis for criterion scores and criterion texts for the development of new formulas and reworking of old ones.
  • Formula discrepancy A look a the discrepancy between the results of different formulas and how writers can benefit from it.(p.25)

Donald Murphy (1947), the editor of Wallace’s Farmer, used a split run with an article written at the 9th-grade level on one run and on at the 6th-grade level on the other run. He found that increasing readability increased readership up of the article 18 percent. In a second test, he took great care not to change anything except readability, keeping headlines, illustrations, subject matter and the position the same. He found readership increases of 45% for an article on nylon and 60% for an article on corn.

Wilbur Schramm (1947) showed that a readable style contributes to the readers’ perseverance, also called depth or persistence, the tendency to keep reading the text. Charles E. Swanson (1948) showed that better readability increases reading perseverance as much as 80 percent. He developed an easy version of a story with 131 syllables per 100 words and a hard version with 173 syllables and distributed each to 125 families. A survey of readers taken 30 hours after distribution showed a gain in the easier version over the hard version of 93% of total paragraphs read, 83% in mean number of paragraphs read, and 82% in the number of correspondents reading every paragraph. Bernard Feld (1948) grouped 101 stories from the Birmingham News into those with high Flesch scores, requiring 9th -grade education or more and those with low scores, requiring less than 9th -grade education. He found readership differences of 20 to 75 percent favoring the low-score versions. Feld’s findings indicated that even a small actual percentage gain for a large-circulation paper greatly increased the number of readers. (p.30)Reading efficiency: Klare, Shuford, and Nichols (1957) followed up these studies with a study of the reading efficiency and retention of 120 male aviators in a mechanics course at Chanute Air Force Base in Illinois. They used two versions of technical training materials, hard (13th-15th grade) and easy (7th
-8th grade).  They measured reading efficiency with an eye-movement camera with which they could determine the number of words read per second and the number of words read per fixation. A strong “set-to-learn” was stimulated by allowing the subjects to re-read the text and giving them a pre-test before the experimental test.

The study showed that the easy text significantly improved both reading efficiency and retention. The results also indicated that a strong “set to learn” improved scores.
Hardyck and Petrinovich (1970) showed the connection between readability and both comprehension and muscle activity in the oral area (subvocalization). Rothkopf (1977) showed the connection between readability and how many words a typist continues to type after the copy page is covered (functional
chaining). (p.30)Readability and course completion. Publishers of correspondence courses are understandably concerned when large numbers of students do not complete the courses. They often suspect the materials are too difficult for the students. Working with Kim Smart of the U. S. Armed Forces Institute, Klare (1973)
applied the Flesch Reading Ease formula to thirty sets of printed correspondence courses used by the military.

They found that two of the high school courses and five of the college courses were too difficult for readers of average or below average reading skill. They then compared their reading analysis to the completion records of the 17 courses that had been in use over two years. They found a Spearman rank-order correlation of .87 between the readability score and the probability of students completing the course. There was a Pearson product-moment correlation of .76.  These results showed the importance of readability for unassisted reading where pressure to complete a course of study is low and competition from distractions is high. (p.30-31)Researchers in readability also addressed the effects of these factors on comprehension:

  • Image words, abstraction, predication, direct and indirect discourse, types of narration, and types of sentences, phrases, and clauses (Gray and Leary 1935).
  • Difficult concepts (Morriss and Holverson 1938, Chall 1958).
  • Idea density (Dolch 1939).
  • Human interest (Flesch 1949, Gunning 1952)
  • Organization (Gunning 1952, Klare and Buck 1954, Chall 1958).
  • Nominalization (Coleman and Blumenfeld 1963; Coleman, 1964)
  • Active and passive voice (Gough 1965, Coleman 1966, Clark and Haviland 1977, Hornby 1974).
  • Embeddedness (Coleman 1966). (p.31)

Among Meyer’s observations are the following:

  • A visible plan for presenting content plays a key role in assessing the difficulty of a text.
  • A plan incorporates a hierarchy showing the dependencies of the facts to one another:
  • The antecedent/consequent plan shows causal relationships in “if/then” logic.
  • The comparison plan presents two opposing views that give weight to both sides.
  • The adversative plan clearly favors one side over the other (political speeches).
The description plan describes the component parts of an item (newspaper articles). This plan is the least effective for remembering and recall.

  • The response plan gives answers to remarks, questions, and problems (science articles).
  • The time-order plan relates events chronologically (history texts). Better readers tend to share the same plan as authors of the material they are reading. Readers who use a different plan other than the authors may be at a disadvantage.
There are two types of highlighting for showing the relationships between items:

  1. Subordination:  used to connect the main idea with supporting text as in a hierarchical structure.
  2. Signaling:  explicit markers to clarify relationships such as:
  • “On the one hand…On the other hand…”
  • “Three things have to be stressed here.”
  • “Thus,” “consequently,” and “therefore”
  • “Nevertheless,” “all the same,” “although,” “but,” and “however”
  • Signaling can also clarify how larger blocks of content are related, for example:
  • “For example,” “For further details,” “summary,” “abstract,” “conclusion,” and “preview.” For more on signaling, see the studies by Jan Spyridakis (1989, 1989a).

Besides reducing the difficulty of the text, Meyer wrote that strategy training can also help older adults deal with the difficulties they encounter in reading.

Also concerned with larger units of text, Bonnie Armbruster (1984) found that the most important feature for learning and comprehension is textual coherence, which comes in two types:
  1. Global coherence, which integrates high-level ideas across an entire section, chapter, or book.
  2. Local coherence, which uses connectives to link ideas within and between sentences. (p.33)

Armbruster found that recalling stories from memory is superior when the structure of the story is clear. She also noted the close relationship between global content and organization. Content is an aspect of structure, and organization is the supreme source of comprehension difficulty.

For local coherence, Armbruster stressed the highlighting that carries meanings from one phrase, clause, or sentence to another:

  • Pronoun references to previous nouns
  • Substitutions or replacements for a previously used phrase or clause
  • (sometimes called “resumptive modifiers”), for example: “These results [previously listed] suggest that…”
  • Conjunctions
  • Connectives
  • Finally, Armbruster supported Kintsch’s finding that coherence and structure are
    more important for younger readers than older ones, simply because they have
    less language and experience. (p.33)

Calfee, Curley, and the familiar outline R.C. Calfee and R. Curleyv(1984) built on the work of Bonnie Meyer. They stressed making the structure of the text clear to upper-grade readers. The content can be simple, but an unfamiliar underlying structure can make the text unnecessarily difficult.

They proposed that the teacher, researcher, and student all need to reach a mutual understanding of the type of outline being used for the text under discussion.

Most students are familiar with the narrative structure, but not with other forms.
Calfee and Curley present a graduated curriculum that enables students to progress from simpler structures to ones that are more difficult:
1. Narrative—fictional and factual
2. Concrete process—descriptive and prescriptive
3. Description—fictional, factual particular, and factual general
4. Concrete topical exposition
5. Line of reasoning—rational, narrative, physical and relational cause-
and-effect
6. Argument—dialogue, theories and support, reflective essay
7. Abstract exposition
  • The lessons of content, organization, and coherence:  Organization and coherence highlight the relationships between words, sentences, paragraphs, and larger sections of text. They enable readers to fit new items of information into their own cognitive systems of organization.
The cognitive studies of readability also showed other problems that texts can
reveal or create, such as:

  • Unfamiliar life experiences and background
  • The need for time to digest illustrations and new material
  • The need for multiple treatments of difficult material
  • The need for learning aids to overcome textual difficulty
The need for learning aids to help readers of different levels of skill.  Generally, however, the cognitive researchers failed to translate their theories into practical and objective methods for adjusting the difficulty of texts for different levels of reading skill. (p.34)Klare concluded that in the studies that showed increased comprehension, transforming text requires attending to other problems besides word and sentence length. “The best assumption, it seems to me,” he wrote, “is that the research workers, probably with considerable effort, managed to change basic underlying causes of difficulty in producing readable versions” (p. 148). Klare then listed the following word-and-sentence variables that affected comprehension:

Word characteristics:
1. Proportion of content (functional) words.
2. Frequency, familiarity, and length of content words.
3. Concreteness or abstractness.
4. Association value.
5. Active vs. nominalized verb constructions.
Sentence characteristics:
1. Length (esp. clause length).
2. Active vs. passive.
3. Affirmative vs. negative.
4. Embedded vs. non-embedded.
5. Low depth vs. high depth (branches).
Since Klare’s 1976 study, there have been other studies showing the positive effects of using formula variables to improve comprehension (Ewing 1976, Green 1979, C. C. Swanson 1979).
In the many studies of before-and-after revision of the text, a negative result does not prove that there is no improvement in comprehension. They show instead that improvement has not been detected. There is a saying in statistics that you cannot prove a negative. Studies reporting a negative result may result from failing to control the reading ability, prior knowledge, interest, and motivation of the subjects. They can also result from failing to control elements of the text such as organization, coherence, and design. The great difficulty of properly conducting such an experiment is seen in following two studies.The Duffy and Kabance study:  Critics worry that technical communicators can too easily misuse the formulas, making documents more difficult, not less (Charrow 1977, Kern, 1979, Selzer 1981, Lange 1982, Duffy 1985, Redish and Selzer 1985, Connaster 1999, Redish 2000, Schriver 2000). These writers offer little or no evidence of such misuse, however, widespread or otherwise. If unscrupulous or careless writers choose to cheat by “writing to the formula” and not attending to other textual issues, careful editors and reviewers easily spot the misuse. The study by Thomas Duffy and Paula Kabance (1981) is a case in point. Because formula critics (e.g., Redish and Selzer 1985; Redish 2000) often
refer to this study, it deserves some attention.

The Duffy and Kabance study consisted of four experiments that examined the effects of changing only word and sentence length on comprehension. It used a “reading to do” task and a “reading to learn” task. The study used four versions of the text. (38)

  • Dumas and Redish (1999), in their work on usability testing, hardly mention reading comprehension. They have us assume that, if test subjects correctly perform a task, they have correctly understood the instructions. When problems arise, however, it is difficult to locate the source of the difficulty.
  • In both usability testing and reading protocols, some subjects are more skilled than others in articulating the problems they encounter. Do problems come from the text or from some other source? If they are located in the text, do they come from the design, style, organization, coherence, or content? We are often left with guesswork and trial-and-error cycles of revision and testing. As experienced writers know, this gets expensive. In preparation for a test, it makes as little sense to neglect the readability of a document as it does to neglect its punctuation, grammar, coherence, or organization. One cannot emphasize enough the importance of testing and of frequent contacts with members of the targeted audience before, during, and after the process of producing documents as urged by Schriver (1997) and Hackos and Redish (1998). Assessing both the reading ability of the audience and the readability of the text will greatly facilitate this process (55)