Something is Wrong with SATII-Chinese

One of the driving forces of the rising popularity in learning Chinese in the US is its recent inclusion as a subject test in SAT II. Our local Chinese school has boasted that three of their students have received perfect scores with others doing pretty well scores-wise. Sounds great, right?

I was skimming through some Chinese BBS today and happened upon someone complaining that their child has also scored perfect on the Chinese SAT II but is in the 57 percentile. What? 43% of the test-takers are receiving perfect scores?!

A quick Google search confirmed this. There are much talk in online forums about this skewed score distribution and its being worthless: here is one example.

Obviously, it's not a good thing to have nearly half of participants scoring perfect in a test of SAT caliber. Something is wrong here. It's a new test, so there must be a lot of kinks to be worked out. But another reason may be that most people who (bother to) take this test are the ones who are or were native Chinese speakers, at least when they were kids. Or they have substantial upbringing in a Chinese-oriented family environment. They dominated the test and they dominated the scores.

It does cheapen significantly the aura of the "perfect score", whatever it's worth. It leaves others embarrassed for making one or two silly mistakes to be dropped way off in the scoring scale. If it continues to be this way, then the question would really be: why bother?

It will be interesting to see if anything is going to be done about this new test.

Our Baseline: 136 Out of 500

So, we have a list of 500 most-frequently-used Chinese characters, it's time to do another test. I copied the characters directly off this page, and pasted them into Excel. Then I stripped off the Pinyin and rearranged into five columns, each for 100 characters. This prints out into three pages.

Last time when I did a character recognizing test with my daughter, I asked her to mark the characters with "+", "check", and "-", for words she definitely knew, she thought she knew but couldn't tell, and not knowing at all. That method was not entirely effective and she got tired and it went on. So this time we tried something simpler: she just checks off any characters she knew.

Here is the result:

1. From 1-100: known 46
2. 101-200: 27
3. 201-300: 15
4. 301-400: 19
5. 401-500: 29

The total is 136 out of the 500.

As noted in the previous post, this kind of tests has several weaknesses, mostly because it asks her to recognize characters without pronunciation help or context. But nonetheless, I think this is as good a baseline measurement as we could get, as far as learning characters go. She certainly knows some more characters that are not in this list, but probably not much. (In that other test, she recognized 100 characters out of about 300, which was a more limited sample.)

We will come back to this later on as one measure of our progresses.

Word Count, the English Style

Just as I have been obsessing with the usage frequency of Chinese characters, this comes along from the Language Log.

But we are getting off the topic here.

We Think We Have A (But Not The) List Now

So this is how things usually work. I posted the report last night, lamenting about not being able to find a list of Most-Frequently-Used characters. This morning my wife promptly sent me a link she had discovered. Lo and be hold, it's a list of 500 Most-Frequently-Used characters! Sorted by their frequencies!! Right here!!!

But hold on. It sounds a little fishy. The author of that page said the list was originally from a TsingHua University study. But he modified the list to "better reflect the oversea Chinese environment". Of course, there is neither a reference to the original list nor any elaboration of what and how he had changed the list himself.

So now I am curious. Taking the hint from that page, I googled the phrase "汉字频度表". This time I did get a lot more hits, including some that seem to refer to the same TsingHua study. But I still could not locate the original study itself. The most important issue for me here is when the study was conducted and how it was sampled. From what I could gather, the study was probably done in the early 80s when TsingHua was involved in creating a Chinese character table (the equivalent of ASCII) for computer processing.

Then I came upon an interesting piece of information. Someone had done a comparison of three different versions of Most-Frequently-Used Chinese characters: The first is an early study done in 1977. The second might be the one we are looking at, simply referred to as "modified version from a TsingHua University Reference cited by Mr. ChenShuYuan". (So I suppose the mysterious origin of this data evaded this author also.). Then yet a third version whose origin this comparison author simply "had forgot", but it was from "the Internet".

Anyway, the comparison does not quite yield a high confidence on the data, as the three versions differ significantly from each other. The ratio of common characters in all three versions is 40% for the top 10 characters, 66% for top 50, 62% for top 100, etc. It gets a little better as the it goes down the list, but still hovering around 80% all the way.

The author of this comparison did a nice job in pointing out the historical difference in sample context between the versions. The first version was done in 1977, which is at the tail end of the Culture Revolution. Its sample had to be skewed by the charged political context at the time. The other two versions, he found, must have been done in the Deng Xiaoping's reform era, as can be seen from the rise and fall of certain characteristic characters (oops, sorry). Indeed, the common ratio between the second and third versions are much closer: around 90%.

It's quite fascinating to look at all this. The stuff I can find here looks like amateur work, but I suppose some linguists must be doing some serious scholastic studies with all the data. It can be safely assumed that none of these belong to the most recent study announced by the government this May.

So, we have a list, or two lists, but not the list. Why is it this hard to find a good list of Most-Frequently-Used characters, with its source and reliability information?

Have You Seen Those 581 Characters?

If you are familiar with the 80-20 rule, you probably know that you don't need to learn a whole lot of Chinese characters before you can read and understand 80% of contents in Chinese. No kidding! Back in May, a Chinese government agency has unveiled, with much fanfare, the result of an extensive study on the "State of Chinese Language" in the media and daily lives.

Among the findings is that merely 581 most-frequently-used characters cover 80% of all contents in Chinese media. Going a little further, if you learn 934 characters, you will understand 90% of Chinese that is out there. With this fact in hand, a Chinese official proudly claimed that it is indeed not difficult to learn Chinese.

I came upon this piece of "news" by looking for a list of most-frequently-used Chinese characters so I can use them to supplement a more primitive test I did with my daughter earlier. I tried for many days and had come up empty. The Internet is really not that great, in Chinese anyway, yet.

There is a table of common Chinese characters available, originally published in 1988. It has 2500 characters grouped by the number of strokes, along with another 1000 of less common ones. (The study in May has found some 300 characters have changed among the 2500 and a correction set is available.)

However, the problem is that this 2500 character table does not come with any individual frequency data so it is impossible to deduct a "more frequently used" subset from them. As a whole, it is simply too large to be of any use for us.

Although the news that 581 characters cover 80% of Chinese content was widely reported on the (Chinese) Internet, there is not a single trace to be found as to what, exactly, those characters are! You can download the official report, along with some of the data, here. But a list of the 581, or 934, is not among them!

Maybe I am not the only one puzzled by the lack of this data in the public domain. A professor had mused before why a usage-frequency list was not included in a standard dictionary. But then I happened upon this fellow by the name of Huang Yong(黄勇), who had done this:

He wrote a little computer program that takes the 2500 common characters in the official list and feeds them into Google search one by one. He then interprets the hits, or search counts, as the character's usage count and sort the characters accordingly.

Personally, I am not sure these search counts could be equivalent to usage. His crude method also has its own flaws, or limitations, as he had documented in his page. But nevertheless, he generates a nice table of characters, sorted in descending order, by their search counts. You can also download his program and regenerate the table at any time.

All right, so it is not scientific. But it is better than nothing. It is probably good enough for our purpose to identify the first 300 or 500 or 900 most-frequently-used Chinese characters anyway.

In the meanwhile, if you happen to see those pesky 581 characters wandering around, drop me a note, okay?

Thanks!

The Stanford Textbook is Here!

The textbook from Stanford Chinese School arrived last night. It has been merely a week since I actually dropped a check in the mail for them, so that is pretty quick. I ordered the second-grade textbook (Books 4, 5, and 6 according to their web site). It actually comes in as a single book, plus three student exercise booklets (two to alternate for odd/even weeks, another one for summer break) and a CD-ROM.

The books are professionally produced, with paper, printing, and binding all in very high quality. Leafing through the pages quickly, I found it to be even more impressive than the samples on the web showed. Here are some quick notes.

What I liked:

1. The emphasize on reading is apparent. The lesson text and reading materials are much longer and with much more substance. They are mostly stories of various interest.
2. Although they tend to be long, there are many, many repetitions (by design) of the same or similar wordings, phrases, and sentence structures throughout the stories. It should make the reading easier as the student progresses.
3. The illustration of grammar usages, the isolation of certain words, phrases, and sentence structures are excellent. (Somehow it reminds me the books I used to crank when I was preparing to take the TOEFL and GRE exams.)
4. Although there seems to be too much homework (four times a week), the assignment is mostly on reading. It does not seem to be too hard or time-consuming.
5. As part of exercising in recognizing characters, most of the homework involves playing a "scramble"-like game: reconstructing the text with a set of loose character cards. The CD-ROM contains a similar exercise. This could be a very fun way for the students to learn characters.
   

What I didn't like:

1. The lessons and readings are way too dominated by fables and ancient stories. While interesting, it lacks variety for an entire year's material.
   
2. It lacks dialogs and conversations that pertain to daily life situations.
3. I would have preferred to have three separate books instead of a single, thick one, as the textbook.
4. There is no teacher's guide. Not even a recommended schedule. From the homework booklets one could deduce that it is supposed to take two weeks for one lesson.
5. For the first lesson (that I have looked at), the homework booklets imply the lesson is taught in two parts, but the CD-ROM homework covers the entire lesson even in the first exercise.
   
6. Something odd: the entire textbook (and homework) does not teach or use Pinyin, as promised. But the homework for summer break contains many Pinyin exercises. An oversight?

Overall, I liked it, a lot. More importantly though, my daughter liked it too. When I showed her the book and how it is designed, she got really, really excited: "I can't wait to get this started!" Well, for that, we shall see. :)

Getting Some Preliminary Numbers

The Stanford Chinese School textbook I evaluated last week has their lists of character vocabulary available online. Mostly out of curiosity, I did a test with my daughter on the lists for their first three books, intended for first grade uses. I asked her to look at each character and see if she had learned them. If she could pronounce one correctly and either give its meaning or using it in a phrase, she would mark it with a "plus". If she felt she knew a character, but could not recall its pronunciation or meaning, then mark it with a "check". And if she had no idea of a character, then mark it with a "minus".

I did not have high expectations on the outcome, as character recognition was one of the difficulties she was experiencing in school. This was also a very unfair test, for the following reasons:

1. She was looking through a "dry" character list without any contextual help. It's much harder to recognize characters this way.
2. There are quite a few characters in the list that are there because they happen to be in the lesson text/story. They are not by any means the must-learn ones for first-graders. My daughter never used this textbook so she should not be expected to know them.
3. There is no Pinyin help in the character list, which is a good thing but it does make it harder for my daughter who is used to see characters with their Pinyin marked.

Anyway, here is a breakdown by the books:

1. First book contains 105 characters, of which she had 50% in pluses, 9% in checks, and 41% in minuses.
2. Second book contains 95 characters, of which she had 28% in pluses, 12% in checks, and 59% in minuses
3. Third book contains 102 characters, of which she had 20% in pluses, 7% in checks, and 75% in minuses

The numbers do not look very bad, against my low expectations. It also shows a nice and telling trend from Book One to Book Three, which probably means the books are well designed in their progress in difficulty.

There are, however, some surprises in the "minus" category: characters I thought she should have learned or at least looked familiar, including
边，毛，黑，红，千，万，共，也，无，长，又，回，林，空，etc.

She was quite interested in doing the test in the beginning, but graduated got bored as it dragged on. At the end, she probably had rushed it without paying close attention. As a third grader, she had quite a fun time adding up her "scores" and trying to figure out the percentages. A little math always helps to brighten up the day.

Along the way, we found a counting error in the Book Three list: it contains 102 characters, not the 105 claimed by the book. I will send an email to its author.

An Example of Awkward Chinese

While looking through my daughter's current Chinese textbook, I noticed this lesson. Here is the text of the entire lesson:

放学了，爸爸在车上告诉我：“下午有一个小朋友来我们家。”我问：“他是谁？”爸爸说：“是云云。”云云是我的好朋友，我听了，真开心。

Literally:

When my school is over, Dad tells me in the car: "A little friend come to our house this afternoon." I ask: "Who is he?" Dad says: "Is Yunyun." Yunyun is my good friend, I hear that, very glad.

(The English version is intentionally awkward.)

In such a simple and short passage, there are one error and two bad ambiguities:

1. Error: There should be a period instead of comma after the sentence 云云是我的好朋友.It's quite a common mistake in Chinese writing to over-use comma through an entire paragraph without periods. But to have that in a textbook is unforgivable.
2. Ambiguity: There is no tense in “下午有一个小朋友来我们家。”. Chinese is not a precise language and tense is very often not enforced. The reader is supposed to deduct the proper tense from context. But here, the ambiguity is quite bad. Within the previous context of Dad telling "me", it seems that Yunyun had already come to our house in the afternoon. Within the later context of "me" being happy, Yunyun is yet to come.
3. Ambiguity: 他 or 她? he or she? The usage of 他 (he) is not explained. In fact, a more natural dialog would be something like "谁来过?"/"谁会来?"("Who came?"/"Who is coming?") or quite simply "谁?" ("Who?").
   

It's not difficult to correct or avoid these simple problems. But having them present in the lesson shows a certain carelessness in the authors of this textbook.

Judging Textbooks by Their Covers

One of the nice things about home-schooling is the flexibility of choosing textbooks on our own, or going without a set textbook at all.

In more recent years, It has become fashionable to learn the Chinese language in the US. Therefore, a wide variety of textbooks targeting the North America market have become available. These could range from the Office of Chinese Language Council International (Hanban) to a free wikibook.

However, it is rather difficult to evaluate these books. Most of the online sellers do not provide adequate samples of their books. When they do provide the text itself, they are shown without the context of any teaching methodology. In a way, we are forced to judge the books by their covers.

Most of these textbooks are compiled and produced from inside China. Although they all claim to aim at the North American audience, their content still has very little, if any at all, relevance of children's life here. This is understandable since their authors, while language experts in China, generally lack life experience here.

The textbook currently being used in my daughter's Chinese school is just such an example. It is one of the earliest and arguably the most widely used in America, published by China's Jinan University in 1997. My daughter found the book uninteresting, which I totally agree. When I look through her second-grade textbook, I found:

• The main text tends to be very simple and naive, even for a second grade book (which is most likely used by a third-grader here), and therefore "uninteresting". There are no stories.
  
• In many occasions the text is awkward, far from the way how a normal kid would talk. Even the dialogs feel like "forced". (This is actually a common problem in Chinese writings, from literature to popular movie/TV dialogs.)
• Pretty much all text characters have their Pinyin printed directly on top of them. This may be helpful for students to learn how to pronounce the characters in the beginning, but it is actually detrimental for them to learn the characters. When they read the text, they tend to read the Pinyin, not the characters.

There is actually another Chinese school in our neighborhood which was established more recently from a split from the older Denver Chinese school. They use a newer textbook called Standard Chinese (标准中文). From this link we can see all the teaching text in this book, but nothing else. The text looks only marginally better than the book we have.

Among the available textbooks, one that is created here in the US stands out as the most interesting. It's author, Dr. Ma Liping, had taught elementary school in China in the past but has then lived and studied in the US, including earning a Ph.D. in curriculum design from Stanford. She had founded the Stanford Chinese School in 1994 and used her own experimental textbook there ever since. Her methodology emphasizes reading and learning Chinese characters through reading. She would postpone learning Pinyin to a later stage so the students are not distracted from learning the characters at the beginning.

The table of content for her second grade textbook seems to show that they have a lot of stories to read, although it's hard to tell if they have anything more closely related to children's lives here at all. There is only one sample available, a variation of the famed fox and crow story. It is certainly an interesting story. The reading material associated with this lesson are not too bad either.

What's most impressive, however, is the way this textbook isolates and illustrates what it calls "grammar point" after the lesson. One of the most difficult and confusing things about Chinese is that the same character can have entirely different meanings at different context and this sample "grammar point" does a pretty good job in showing and teaching them. (It even uses a little English at that!)

The Stanford Chinese School web site also has some articles indicating the textbook is gaining popularity in other Chinese schools. They also point out that many schools have suffered from various difficulties in the attempts to adopt them.

I also have my own doubts. While emphasizing reading, this textbook seems to assume that the students already possess sufficient oral communication skills in Chinese, perhaps from their family environment. That would not be true for my daughter, who so far speaks English exclusively at home. It is also not clear how they teach pronunciation without using Pinyin. (My daughter has already learned Pinyin, or its concept, so this may be a moot point for us.)

But it does seem to be worthwhile to invest $65 for a closer look at one-year's worth of textbook. So off goes a check. We should know more soon enough.

What Kids Need to Learn in Chinese Language -- in China

I was looking for some measuring sticks on learning Chinese when I found that a new Curriculum Standard for Chinese is being implemented in China in the last few years. The standard spells out the goals of Chinese language education during the 9-year (K-9) compulsory education period.

The goals are a mixture of political and cultural aptitudes, general reading and writing skills, and some quantified measurements. These are further divided into four periods: Grades 1-2, 3-4, 5-6, and 7-9. Although my daughter is currently in the 3rd grade, her Chinese level would probably fall into the first period, i.e., for Grades 1-2.

According to the Standard, the goals for Grades 1-2 are:

• Learning the Chinese characters:

1. Able to enjoy learning characters
2. Able to recognize 1600 to 1800 commonly-used characters; able to write 800 to 1000 of them
3. Knowing the basic strokes, writing with the correct stroke order and spacing; able to appreciate the beauty of a character's shape
   
4. Having a good posture and habit while writing; writing neatly
   
5. Complete the learning of Pinyin: able to correctly pronounce the initials, finals, and tones; able to write them correctly
6. Able to pronounce new characters by their Pinyin, able to look up new characters in a dictionary by their Pinyin and/or strokes
   

• Reading

1. Able to enjoy reading
2. Able to read correctly and smoothly with the standard Mandarin
3. Able to read silently, without the help of voice or finger tracing
4. Able to read with the help of pictures
5. Able to understand phrases with the help of their context
6. Able to read simple fairy tales, fables, and stories
7. Able to read children's poems and other simple poems, enjoy the beauty of language
   
8. Knowing the common punctuations and their usage in different expressions
9. Able to recite 50 poems or proses, read more than 50,000 character's worth of books outside of the classroom
10. Love books and take good care of books
    

• Writing

1. Interested in writing sentences and writing down one's own words, imaginations and feelings
2. Enjoy using new words when writing sentences
3. Able to use punctuations correctly
   

An Interview with my Daughter

What do you like about Chinese school?

The break time, when we get a break from listening to the teacher all the time and walk around and play a bit. I also like seeing my friends there.

What do you not like about Chinese school?

I can't understand what the teacher says most of the time. I also don't like that I have to go to Chinese school when I may want to do something else at the time.

What do you like about learning the Chinese language?

If I go to China, I will be able to talk to my relatives. Even here in the US, I will be able to understand when people around me talking in Chinese.
It's also good to learn another language, especially one my Mom and Dad speak.
When I meet new friends here, some of them would ask me to speak something in Chinese (or Japanese, when they mistook me as a Japanese), I would be able to say something for them.

What do you not like about learning the Chinese language?

It just takes a lot of time.

What is a typical class in your Chinese school like?

We start with a new lesson. The teacher reads for us first. She then tells us to read it ourselves. She may ask some students to recite it individually.
Then she gives us a sheet of paper with new words we are learning for this class. We learn the words by their structures. She may also explain the meaning of the words but I can't understand that.
We then do some exercises and read other parts of the textbook.
Finally, we get our homework.

How do the students talk to the teacher?

Usually the teacher asks questions. This is actually pretty often. Students would raise their hands to answer. Some students raise hands more often than others.
Sometimes the teacher would ask me to answer the question even when I didn't raise my hand. I would just stay quiet and she would move on. This happens to other students too.
Sometimes I know the answer but I still say nothing because I don't feel comfortable. The teacher moves on so fast.

How many students are in your class?

About 20, and most of them can understand the teacher well and answer questions.

Do you talk to each other in Chinese?

Not usually, even those who are good in Chinese don't talk to each other in Chinese.

What do you think about the textbooks?

They are okay, but not really interesting. Nothing I really like, nothing I don't like.

Do you like to read the text aloud?

Yeah, I like to do that at home. But in school I don't like to do that alone. I am not comfortable about my pronunciation. Or sometimes I haven't had the text memorized.

What kind of things would you like to have more in your textbook?

More things close to what I talk and do every day; stuff I would like to talk about, like food, movies, TV, etc.
More activity-like exercises, not just reading;

What do you like about homework?

I like it when it has a lot of different kinds of exercises.

What do you not like about homework?

I don't like having to memorize text. I don't like having to write a sentence on my own using different words, or copy the same words 9 times.

How many new words do you usually learn each week?

About 10. Most of them I would forget soon after.

What's the most difficult thing about learning new words?

Usually the meaning. I can get the pronunciation and structure from the class but I don't understand it when the teacher explains their meaning.

Do you like to read Chinese books?

No. Mainly because I don't know most of the words. It's hard to understand.

Do you like to talk to people in Chinese?

No. I don't know what to say. We never learned how to start a conversation. I also don't know many words.

If we learn how to have a conversation in Chinese, would you like to talk to people in Chinese?

Maybe.

Are you glad that you don't have to go to Chinese school when we start homeschooling?

Yeah. I am always glad when the Chinese school is not in session anyway.

How do you feel about this homeschooling?

I feel it will be better. I will be able to understand more when the instruction is in English. I will be more comfortable in saying what I don't understand.

So, Why Are We Doing This?

To start it off, some background information:

My daughter was born in the USA and is now eight years old. She had visited China only once last year, for three weeks. Both her parents were born in China but have lived in the USA for close to 20 years. Although her parents do speak Chinese to each other at home most of the time, she grew up speaking English exclusively. In fact, she was in daycare before her second birthday and has been "immersed" in English ever since. She is now a third-grader in our local public school.

Up till now, she had been going to our local Chinese School for two and half years, passing through the school's preschool, PinYin, and first-grade language classes. She is now in the second-grade language class.

She had never liked the Chinese school. This is not surprising since if you poll the kids in school you probably will get over 90% of them saying they dislike or even hate the school. They are there simply because their parents made them to. In the earlier years, we were able to bribe her into going with incentives that she got to see her best friend (who is no longer living in our neighborhood) and that she could also do dancing class there, etc.

But since the end of last year, she started to use the H-word, as in "I hate Chinese School".

The most significant difficulty she is encountering in school, according to herself, is that she could no longer follow the class instructions, which are entirely in Chinese. Her Chinese language level is just not good enough for the class. So she had to sit in the class for two hours staring at a teacher and not understanding anything she says. If she had learned any Chinese at all now, she says, it is all from doing homework at home with her Mom's help. So, in essence, we have already been homeschooling her with the school merely playing the role of providing weekly study materials in the form of homework assignments.

As most of the kids today, my daughter has a busy schedule for evenings and weekends. As she grows older, it is and will be getting worse. Her soccer and swimming, for example, now has practices or games three times a week when in season. She is also learning piano regularly and participating in other activities when available. Therefore, spending two hours in Chinese school every Sunday afternoon becomes a pretty big investment in her time. When the benefit could not justify the cost, we have to seek alternatives.

Equally important is her attitude. It's not going to be a big leap from "I hate Chinese school" to "I hate Chinese" (the language, of course). Right now, she is still geniurely interested in learning the language, something we need to nurture and protect.

Hello World

Finally, we decided to teach my daughter the Chinese language at home instead of sending her to the Denver Chinese School every Sunday. Me and my daughter are starting this blog together to record this experiment and adventure.