That is the format I use for my vocab and content quizzes (ELA teacher here). But the last thing ELA teachers should do, IMO, is rely on quizzes or tests for the bulk of their grades. I use them often, but they aren't weighted very highly. I look for how students do on immediate reader responses, essays, group presentations, things of that sort that require the student to be the source. And for each assignment I have a list of possible topics/projects/ideas to choose from. It's painfully easy to catch when a student is cheating this way. So easy that it almost never happens because even the students know they'll get caught.aerius wrote:Teacher makes up 2-4 different versions of the test. Numbers are changed or question order is different. Good luck cheating when no one beside you has the same test as you. One of my high school science teachers did this to us all on all of his tests, and he'd walk around the room during the tests to ensure that everyone had their eyes on their own papers. Anyone caught trying to cheat had his test ripped up on the spot. After the first couple tests got shredded, no one even thought about trying to copy answers from others.ArmorPierce wrote:Okay guys question. Without standardized tests how do you differentiate between those who actually know the material relative to those who just copy all the answers and works from others?
Should parents legally pay for their ADULT children's tuitio
Moderators: Alyrium Denryle, Edi, K. A. Pital
Re: Should parents legally pay for their ADULT children's tu
"A person is smart. People are dumb, panicky, dangerous animals and you know it. Fifteen hundred years ago everybody knew the Earth was the center of the universe. Five hundred years ago, everybody knew the Earth was flat, and fifteen minutes ago, you knew that humans were alone on this planet. Imagine what you'll know...tomorrow."
-Agent Kay
-Agent Kay
Re: Should parents legally pay for their ADULT children's tu
I think perhaps people should disentangle two concepts here.
The first is standardised testing. This is a necessary condition to make a national level qualification meaningful.
The second is the (apparently sole method of testing in the US): multiple guess tests. Other countries manage to have essay and long answer exams which are sat by every pupil and marked to the same standard, and give results that are not just pass/fail.
Standardised testing doesn't have to be a farce.
The first is standardised testing. This is a necessary condition to make a national level qualification meaningful.
The second is the (apparently sole method of testing in the US): multiple guess tests. Other countries manage to have essay and long answer exams which are sat by every pupil and marked to the same standard, and give results that are not just pass/fail.
Standardised testing doesn't have to be a farce.
Apparently nobody can see you without a signature.
- Elheru Aran
- Emperor's Hand
- Posts: 13073
- Joined: 2004-03-04 01:15am
- Location: Georgia
Re: Should parents legally pay for their ADULT children's tu
There are *some* essay and long-answer tests out there. The SAT has a section where you write out answers. Generally this kind of thing is in tests upon English/grammar and literature, though. Math and sciences are almost always multiple-choice. Math tests will generally have a section to the side of the problems where you can scratch out the solution, but that doesn't really matter to the answer that you put down on the sheet.
And results aren't pass/fail for the most part. They do have degrees of graduation. Some tests more so than others.
And results aren't pass/fail for the most part. They do have degrees of graduation. Some tests more so than others.
It's a strange world. Let's keep it that way.
-
- Emperor's Hand
- Posts: 30165
- Joined: 2009-05-23 07:29pm
Re: Should parents legally pay for their ADULT children's tu
Yeah... and note that you CAN'T do that on one of the tamper-proof, heavily policed, procedure-must-be-followed standardized tests handed down from state and county levels.aerius wrote:Teacher makes up 2-4 different versions of the test. Numbers are changed or question order is different. Good luck cheating when no one beside you has the same test as you. One of my high school science teachers did this to us all on all of his tests, and he'd walk around the room during the tests to ensure that everyone had their eyes on their own papers. Anyone caught trying to cheat had his test ripped up on the spot. After the first couple tests got shredded, no one even thought about trying to copy answers from others.
The student can rip up their test in a fit of pique and ignore the consequences to themselves. But you can't rip up a test you think is the result of cheating, because that would be prejudicial to the student.
The teachers at a high school also (in my case) cannot flunk students on the spot or even GRADE them for taking any county/state mandated assessment. Also, we can't just throw them out of the room, there's a procedure which then becomes a distraction and timewaster. And, if you're trying to deal with something nondisruptive,Same technique was used in many of my university courses, except there were up to 8 versions of the test depending its importance and the size of the class. Cheating was pretty much impossible, anyone who got caught was flunked on the spot and kicked out of the room.
Well, even standardized tests that aren't multiple-choice can become a problem if they are, for example, disruptive of the learning environment. Or if they promote an attitude of contempt for tests because the state diagnostic tests are consequence-free as far as the students are concerned.Steel wrote:I think perhaps people should disentangle two concepts here.
The first is standardised testing. This is a necessary condition to make a national level qualification meaningful.
The second is the (apparently sole method of testing in the US): multiple guess tests. Other countries manage to have essay and long answer exams which are sat by every pupil and marked to the same standard, and give results that are not just pass/fail.
Standardised testing doesn't have to be a farce.
But yes, it is very much possible to do a good job with this. What you do is, you have one state mandated test at the beginning of the year, and one at the end. These tests have consequences. If a teacher does not bring about enough improvement from the start of the year to the end, you help them. They are only punished if there is evidence that they're violating some basic rule of professionalism.
In the middle you offer tests that may or may not be state-designed but are NOT major, mandatory events that require constant use of security and hyperanalysis where teachers can get fired for 'screwing up' and having unsatisfactory scores.
Even a multiple choice test can be nontoxic, if it's used with the right attitude and ancillary supports.
This space dedicated to Vasily Arkhipov
- ArmorPierce
- Rabid Monkey
- Posts: 5904
- Joined: 2002-07-04 09:54pm
- Location: Born and raised in Brooklyn, unfornately presently in Jersey
Re: Should parents legally pay for their ADULT children's tu
If you are giving out tests based on the concepts taught in the class room, aren't you teaching to the test? How is this different from teaching to the test for a standardized test rather than a teacher provided test? The teacher has more control of what they teach yes but then you throw out the entire concept of standardized education and objectivity.
I think that the standardized test results should be considered along with other factors. They might not be officially but I think that they are considered when developing a student that may be having issues.
I think that the standardized test results should be considered along with other factors. They might not be officially but I think that they are considered when developing a student that may be having issues.
Brotherhood of the Monkey @( !.! )@
To give anything less than your best is to sacrifice the gift. ~Steve Prefontaine
Aoccdrnig to rscheearch at an Elingsh uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht frist and lsat ltteer are in the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae we do not raed ervey lteter by it slef but the wrod as a wlohe.
To give anything less than your best is to sacrifice the gift. ~Steve Prefontaine
Aoccdrnig to rscheearch at an Elingsh uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht frist and lsat ltteer are in the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae we do not raed ervey lteter by it slef but the wrod as a wlohe.
- ArmorPierce
- Rabid Monkey
- Posts: 5904
- Joined: 2002-07-04 09:54pm
- Location: Born and raised in Brooklyn, unfornately presently in Jersey
Re: Should parents legally pay for their ADULT children's tu
Are you guys also against SATs, GMAT, GRE, and professional qualification exams?
Brotherhood of the Monkey @( !.! )@
To give anything less than your best is to sacrifice the gift. ~Steve Prefontaine
Aoccdrnig to rscheearch at an Elingsh uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht frist and lsat ltteer are in the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae we do not raed ervey lteter by it slef but the wrod as a wlohe.
To give anything less than your best is to sacrifice the gift. ~Steve Prefontaine
Aoccdrnig to rscheearch at an Elingsh uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht frist and lsat ltteer are in the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae we do not raed ervey lteter by it slef but the wrod as a wlohe.
- Elheru Aran
- Emperor's Hand
- Posts: 13073
- Joined: 2004-03-04 01:15am
- Location: Georgia
Re: Should parents legally pay for their ADULT children's tu
Giving tests based upon concepts taught in the classroom isn't teaching the test as much as it is making sure that your students understand the material. Teaching the test happens when the test is more important than the material itself for advancement.
And 'standardized education' is somewhat of a misnomer. True, the majority of kids are 'average' and you can teach basically the same stuff, but there's enough individuality out there that you just can't really expect them to all conform to the same identical metric, which is what the standardized tests tend to force them into.
SAT's, GRE, professional exams are a different category of standardized test. SAT and GRE are college entrance exams, and while they're important, they're not the only metric that colleges use, unlike standardized tests for advancement in primary education. Professional exams are also important as they're usually better done than standardized tests-- they often require some degree of demonstration of the material as opposed to simply checking off answers. Take the bar exam; it's largely a bunch of essay questions, so you have to actually write out your answers, which are then actually read by a person who has to judge the quality of your answer and compare it to the "real" answer. This is especially important as different approaches to the same answer may be taken.
And 'standardized education' is somewhat of a misnomer. True, the majority of kids are 'average' and you can teach basically the same stuff, but there's enough individuality out there that you just can't really expect them to all conform to the same identical metric, which is what the standardized tests tend to force them into.
SAT's, GRE, professional exams are a different category of standardized test. SAT and GRE are college entrance exams, and while they're important, they're not the only metric that colleges use, unlike standardized tests for advancement in primary education. Professional exams are also important as they're usually better done than standardized tests-- they often require some degree of demonstration of the material as opposed to simply checking off answers. Take the bar exam; it's largely a bunch of essay questions, so you have to actually write out your answers, which are then actually read by a person who has to judge the quality of your answer and compare it to the "real" answer. This is especially important as different approaches to the same answer may be taken.
It's a strange world. Let's keep it that way.
Re: Should parents legally pay for their ADULT children's tu
Back in the 90s Dark Ages when I went to high school, before the common core, we basically had the SATs and the Regents (a New York state standardized test) as the most significant testing milestones. Despite the overhauls introduced by "No Child Left Behind" and the "Common Core" curriculum, the end-result was pretty similar. The American education system is simply optimized for Scantrons, or whatever equivalent they use now. We were even explicitly taught ways to "game" the system without really knowing the material, e.g. look for answers which are obviously wrong, rule them out, and then choose between the remaining ones. Also, almost any time there was an option like "Both A & B", it was usually the correct answer.Simon Jester wrote:That's the world I live in. It may not be the same world you think modern American education lives in, but it kind of is. And as a result we're turning out a generation where, aside from a handful of the very cleverest, nearly everyone is educated to fill out multiple choice questions- i.e. not educated very thoroughly.
- Ziggy Stardust
- Sith Devotee
- Posts: 3114
- Joined: 2006-09-10 10:16pm
- Location: Research Triangle, NC
Re: Should parents legally pay for their ADULT children's tu
Seriously?aerius wrote:Same technique was used in many of my university courses, except there were up to 8 versions of the test depending its importance and the size of the class. Cheating was pretty much impossible, anyone who got caught was flunked on the spot and kicked out of the room.
At most universities, at least here in the US, a professor could get in a lot of trouble (possibly fired) for kicking a student out of the room or tearing up a test or anything so flamboyant/dramatic. Typically, academic institutions have very specific policies in place for how to handle academic dishonesty (whether cheating or plagiarism), and it is very bad form not to follow them to the letter. Part of it is a function of due process (the school doesn't want students to be punished unless the cheating is proven; otherwise you open up the possibility of professors arbitrarily punishing students they don't like), and part of it is a function of privacy rights. While academic dishonesty hearings/investigations are in process, professors often aren't allowed to talk about it at all, or even mention that they have a student being investigated. I personally know people working in academic institutions who have gotten in trouble for even accusing a student of cheating to their face rather than going through the proper channels. Often, the professor never directly interacts with the student during the process; it is all done by an independent committee.
- ArmorPierce
- Rabid Monkey
- Posts: 5904
- Joined: 2002-07-04 09:54pm
- Location: Born and raised in Brooklyn, unfornately presently in Jersey
Re: Should parents legally pay for their ADULT children's tu
Well something that I do like about the standardized tests is that it has a factor of equalization. Without standardized tests people put more wheight on prestige of the institution that you came from and how academically rigourous it is whether it is actual or preceived (for instance, people going to ivy league schools).
Standardized testing puts everyone on a equal playing field. Oh you went to a ivy league prestigious school? Well that doesn't matter because we scored the same on the standardized test.
Without standardization and objectivity, people put more weight on preceived prestige, which unlike standardized testing, you are probably born to with little that you can do about.
Standardized testing puts everyone on a equal playing field. Oh you went to a ivy league prestigious school? Well that doesn't matter because we scored the same on the standardized test.
Without standardization and objectivity, people put more weight on preceived prestige, which unlike standardized testing, you are probably born to with little that you can do about.
Brotherhood of the Monkey @( !.! )@
To give anything less than your best is to sacrifice the gift. ~Steve Prefontaine
Aoccdrnig to rscheearch at an Elingsh uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht frist and lsat ltteer are in the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae we do not raed ervey lteter by it slef but the wrod as a wlohe.
To give anything less than your best is to sacrifice the gift. ~Steve Prefontaine
Aoccdrnig to rscheearch at an Elingsh uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht frist and lsat ltteer are in the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae we do not raed ervey lteter by it slef but the wrod as a wlohe.
- Elheru Aran
- Emperor's Hand
- Posts: 13073
- Joined: 2004-03-04 01:15am
- Location: Georgia
Re: Should parents legally pay for their ADULT children's tu
That is a good point about them, and I agree.ArmorPierce wrote:Well something that I do like about the standardized tests is that it has a factor of equalization. Without standardized tests people put more wheight on prestige of the institution that you came from and how academically rigourous it is whether it is actual or preceived (for instance, people going to ivy league schools).
Standardized testing puts everyone on a equal playing field. Oh you went to a ivy league prestigious school? Well that doesn't matter because we scored the same on the standardized test.
Without standardization and objectivity, people put more weight on preceived prestige, which unlike standardized testing, you are probably born to with little that you can do about.
Unfortunately it only really applies to higher level tests such as the bar exam or college entrance tests. The problem with standardized tests is primarily at the grade-school level, which is where they're doing the most harm.
And prestige still counts for a lot. Theoretically, a bachelor's degree is the same anywhere you go (provided you earned it from an appropriately accredited school). In practice, people are going to respect a bachelor's degree from, say, University of Georgia, over one from Armstrong State University in Savannah. Part of that is history-- UGA started out as a university and has state-wide reach, while Armstrong had its origins as a community college and remains largely a local institution-- but the biggest part of it is that UGA is simply better known than Armstrong.
When you throw schools like Ivy League institutions, massive state universities, etc, into the mix, people who graduated with degrees from those schools will have more prestige than those who graduate from smaller or less well known schools, regardless of whether it's legitimate or not. A lawyer who went to Harvard and got passing grades on the bar exam is still going to have a bigger cachet than one who got their degree from Iowa State University and got top-notch grades on the bar.
It's a strange world. Let's keep it that way.
-
- Emperor's Hand
- Posts: 30165
- Joined: 2009-05-23 07:29pm
Re: Should parents legally pay for their ADULT children's tu
Except that at the level where all the teachers are complaining about the tests, it doesn't put everyone an an equal playing field. The kids whose parents can afford to pay thousands of bucks for SAT prep classes will consistently outperform the kids who don't have that advantage, even if they're equally intelligent and pay the same amount of attention in school.ArmorPierce wrote:Standardized testing puts everyone on a equal playing field. Oh you went to a ivy league prestigious school? Well that doesn't matter because we scored the same on the standardized test.
To compensate, the school can offer SAT prep for free... in place of normal math and English instruction. And SAT questions are like a weird screwy parodyland version of what it takes to show real knowledge of math and English. Very few SAT-testing skills transfer to general knowledge and education. So the more time they spend coaching you on how to do well on the SAT... the less you know. In absolute terms.
But if the school doesn't accept that its students will learn less and therefore coach them on the SAT... well, the privileged kids will do better on the SAT than the less privileged kids.
This space dedicated to Vasily Arkhipov
- ArmorPierce
- Rabid Monkey
- Posts: 5904
- Joined: 2002-07-04 09:54pm
- Location: Born and raised in Brooklyn, unfornately presently in Jersey
Re: Should parents legally pay for their ADULT children's tu
Yes SAT prep will help, as will have a personal tutor after school for all the classes. Don't quote me on this but I believe I heard that the essay questions were being removed from the SAT to partly address this.
That said, doing SAT prep classes does require actual work. I'd far rather the opportunity to achieve on this test through hard work that is objectively measured. Although requiring more self motivation, self-study is definitely an option, and a much cheaper one. There are prep books at the library and far cheaper self study resources available to purchase. The internet has also tremendously leveled the playing field in the last few years even compared to a decade ago. It is mind blowing easy to google a concept that you don't understand these days and read up and figure it out.
I credit my success to self study and an interest in learning, although when I was younger it didn't have purpose or direction. Even now I make use of self study material for professional test that I am taking.
I know it's using myself as an example is an anecdote but here you go. I am like a poster child disadvantaged upbringing. Both my parents are immigrants and don't speak english (I was put in a speech class when I started school, my brother was put in ESL[english as secondary language] where he was for several years until we moved, despite him not knowing how to speak any other language outside english], no motivation or direction at home and lots of drama, constant moving every couple of years, grew up in poverty in burned out urban area. Yet despite all this I was able to due to make progress due to my standardized test scores demonstrating my abilities in an objective manner. I mean if I did without trying despite all the obstacles in my way. I don't get what's so difficult about it.
For the record, I think that standardized testing shouldn't be the only metric, but it should definitely be an important tool to be considered.
That said, doing SAT prep classes does require actual work. I'd far rather the opportunity to achieve on this test through hard work that is objectively measured. Although requiring more self motivation, self-study is definitely an option, and a much cheaper one. There are prep books at the library and far cheaper self study resources available to purchase. The internet has also tremendously leveled the playing field in the last few years even compared to a decade ago. It is mind blowing easy to google a concept that you don't understand these days and read up and figure it out.
I credit my success to self study and an interest in learning, although when I was younger it didn't have purpose or direction. Even now I make use of self study material for professional test that I am taking.
I know it's using myself as an example is an anecdote but here you go. I am like a poster child disadvantaged upbringing. Both my parents are immigrants and don't speak english (I was put in a speech class when I started school, my brother was put in ESL[english as secondary language] where he was for several years until we moved, despite him not knowing how to speak any other language outside english], no motivation or direction at home and lots of drama, constant moving every couple of years, grew up in poverty in burned out urban area. Yet despite all this I was able to due to make progress due to my standardized test scores demonstrating my abilities in an objective manner. I mean if I did without trying despite all the obstacles in my way. I don't get what's so difficult about it.
For the record, I think that standardized testing shouldn't be the only metric, but it should definitely be an important tool to be considered.
Brotherhood of the Monkey @( !.! )@
To give anything less than your best is to sacrifice the gift. ~Steve Prefontaine
Aoccdrnig to rscheearch at an Elingsh uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht frist and lsat ltteer are in the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae we do not raed ervey lteter by it slef but the wrod as a wlohe.
To give anything less than your best is to sacrifice the gift. ~Steve Prefontaine
Aoccdrnig to rscheearch at an Elingsh uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht frist and lsat ltteer are in the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae we do not raed ervey lteter by it slef but the wrod as a wlohe.
-
- Emperor's Hand
- Posts: 30165
- Joined: 2009-05-23 07:29pm
Re: Should parents legally pay for their ADULT children's tu
That would make things worse, not better. Essay-writing at least has some real overlap with the skills you'd normally get taught by someone who gives zero fucks about the SAT and is actually trying to make you well rounded and educated. It also has overlap with the skills you need in college and in the workplace, because being able to represent a coherent opinion in writing is valued in those places.ArmorPierce wrote:Yes SAT prep will help, as will have a personal tutor after school for all the classes. Don't quote me on this but I believe I heard that the essay questions were being removed from the SAT to partly address this.
Whereas no one, nowhere, really cares about your ability to answer multiple choice questions quickly.
If they were really revamping the SAT to be an accurate test of students' raw problem-solving skills, rather than their skill at cramming and practicing arbitrary, contrived classes of problems that haven't changed significantly in twenty years, then there would be nothing BUT essays and extended-response questions on the test.
Thing is, if X units of work in a dedicated SAT prep class improves your SAT score by 50 points, while X units of work in a class teaching trigonometry only improves your SAT score by 15 points... does that mean that trigonometry is less important than taking more SAT prep? Or that a student who spent the time on SAT prep is an objectively better student?That said, doing SAT prep classes does require actual work. I'd far rather the opportunity to achieve on this test through hard work that is objectively measured.
[Before you answer, reflect that trigonometry is deemed 'too advanced' to be required on the SAT. And yet basically all colleges expect that any student with a scrap of mathematical or technical aptitude will have taken one or more classes that use it before graduating from high school]
So in short, as an 'objective measure of hard work' the SAT sucks, because even on subjects it purports to measure (math skills), it doesn't measure all high school math skills. It only measures some of them. And even the ones it does measure, it measures in a very specific, stilted way. So there's an incentive to game the system by learning "test-taking strategies" that have nothing to do with how much actual reading, writing, and mathematics you know.
So, basically, a student who self-studies and practices a thousand SAT questions will hopefully get the same benefit as Richard J. Butterworth VI gets from their tutors. See, I believe that... but the problem is that the SAT clearly isn't a fair or objective test.Although requiring more self motivation, self-study is definitely an option, and a much cheaper one. There are prep books at the library and far cheaper self study resources available to purchase. The internet has also tremendously leveled the playing field in the last few years even compared to a decade ago. It is mind blowing easy to google a concept that you don't understand these days and read up and figure it out.
It doesn't matter that in theory, a person who has nobody to show them the tips and tricks for beating the SAT can do as well as a person who has thousand dollar tutors to show them how. The point is that the people who can splurge a thousand for a tutor get advantages over the ones who don't. It's not as level a playing field as you make it out to be.
And if you wanted to make the test more objective you'd have to make the questions more challenging, rigorous, and complicated... which would mean they would NOT look like the old familiar multiple choice tests we're used to in America.
Thing is, it's not like you can't self-study for things that aren't standardized tests. In fact, virtually all professionals are expected to self-study all sorts of topics all the time; it's called "doing background research." Or "figuring out how to use that damn software IT just upgraded us to."I credit my success to self study and an interest in learning, although when I was younger it didn't have purpose or direction. Even now I make use of self study material for professional test that I am taking.
The fact that it is possible to study for tests does not make the tests good, or for that matter bad. It's normal, it's part of life, it's what people really mean when they say "you never stop learning."
Well, I'd appreciate it if you'd stop to seriously understand what the people around you are saying. Here are my concerns about standardized tests in primary and secondary education:I know it's using myself as an example is an anecdote but here you go. I am like a poster child disadvantaged upbringing. Both my parents are immigrants and don't speak english (I was put in a speech class when I started school, my brother was put in ESL[english as secondary language] where he was for several years until we moved, despite him not knowing how to speak any other language outside english], no motivation or direction at home and lots of drama, constant moving every couple of years, grew up in poverty in burned out urban area. Yet despite all this I was able to due to make progress due to my standardized test scores demonstrating my abilities in an objective manner. I mean if I did without trying despite all the obstacles in my way. I don't get what's so difficult about it.
For the record, I think that standardized testing shouldn't be the only metric, but it should definitely be an important tool to be considered.
____________________________________
1) The tests are not rigorous enough, or are not rigorous in the right ways.
Instead of asking challenging questions that require detailed, thoughtful responses reflecting useful skills, most standardized tests in America are full of trivial multiple choice questions that exaggerate the importance of hurrying up and the power of guesswork.
2) The tests are too numerous.
Schools have to teach as well as test. The number of tests per year has to be limited so that there is enough time to actually teach students, and so that they have time to recharge their batteries between tests rather than getting hit with six standardized tests in a row every couple of months. There comes a point at which using these tests to collect data is causing more disruption of the learning environment than it can justify.
3) Many disadvantaged students do not take the tests seriously
They've decided they can blow off such tests, and because of the lack of consequences for failure they get away with it. The reason failure has no consequences is that most of the tests are being used to collect information on teachers and students, not to assess whether a student should pass the class and move on to the next grade or subject.
So if you spend lots of time (see 2) taking tests that are difficult (because you're below grade level) but where failing and guessing over and over has no consequences... you develop very BAD test-taking habits, which make things worse when you finally get to a test that matters and blow it off rather than studying, preparing, and thinking through your answers.
4) The tests are too high-stakes for the schools
This creates too much incentive for schools to cheat, which in turn creates an arms race escalation of the security on the tests... which makes them more disruptive of the learning environment. It also means that a struggling school gets dismantled rather than helped if its test scores drop too low. Which is not a good way to make sure the actual outcomes for students improve.
5) The tests do not align well with what the content area ought to TEACH
This is not always true but it sometimes is. For example, a test of your English language skills that doesn't require you to know how to write a short essay is not a good test. Writing is fundamental to your ability to make yourself understood in any language.
____________________________________
So, notice I didn't say "abolish standardized tests." What I did say was "the tests we have are seriously flawed in many ways, which hurts our ability to give students the best possible education AND makes the results of the test less useful and more politicized than they ought to be."
This space dedicated to Vasily Arkhipov
- ArmorPierce
- Rabid Monkey
- Posts: 5904
- Joined: 2002-07-04 09:54pm
- Location: Born and raised in Brooklyn, unfornately presently in Jersey
Re: Should parents legally pay for their ADULT children's tu
Apologies for delay in response. Busy times. I'll make it quick.
Now you bring up another interesting point. Is the point of school the general teaching of academic knowledge or is it for teaching you to be a well rounded adult (I think it should be for a well rounded adult. Social skills is very important and adult day to day interaction. Would we grade someone who spends all there time lower for lack of day-to-day social skills?
[Before you answer, reflect that trigonometry is deemed 'too advanced' to be required on the SAT. And yet basically all colleges expect that any student with a scrap of mathematical or technical aptitude will have taken one or more classes that use it before graduating from high school]
So in short, as an 'objective measure of hard work' the SAT sucks, because even on subjects it purports to measure (math skills), it doesn't measure all high school math skills. It only measures some of them. And even the ones it does measure, it measures in a very specific, stilted way. So there's an incentive to game the system by learning "test-taking strategies" that have nothing to do with how much actual reading, writing, and mathematics you know.[/quote]
It is my opinion that more complex problems should be left for the class room and class room tests. The point of standardized test is an objective general assessment of skills that you take away with you and accumulate over the years, not something that you will forget right after taking the class. The CPA exam for example tests on a large breath of concepts but does not go to deeply into any one like you might takle in a class.
It doesn't matter that in theory, a person who has nobody to show them the tips and tricks for beating the SAT can do as well as a person who has thousand dollar tutors to show them how. The point is that the people who can splurge a thousand for a tutor get advantages over the ones who don't. It's not as level a playing field as you make it out to be.
And if you wanted to make the test more objective you'd have to make the questions more challenging, rigorous, and complicated... which would mean they would NOT look like the old familiar multiple choice tests we're used to in America.[/quote]
There are definitely advantages to a tutor, I don't disagree, but I feel that those advantages are more readily impacted in a day-to-day class room environment were the tutor is able to help on day-to-day tasks. A standardized general assessment test is a pretty personal journey that someone has to take and drill into their head.
How does making a test more challenging and rigorous and complicated make it more objective? Objectivity comes from the fact that it is standardized and you are taking human grader judgement out of the picture.
Sorry ran out of time will get to your other points later. But I will admit I don't have too much background on the standardized public school assessments themselves, I'm just pro standardized test in general.
I can be mistakened but I beleive that a lot of it was due to foreign test takers were disadvantaged in taking the Essay and it felt that it favored certain demographicSimon_Jester wrote:That would make things worse, not better. Essay-writing at least has some real overlap with the skills you'd normally get taught by someone who gives zero fucks about the SAT and is actually trying to make you well rounded and educated. It also has overlap with the skills you need in college and in the workplace, because being able to represent a coherent opinion in writing is valued in those places.ArmorPierce wrote:Yes SAT prep will help, as will have a personal tutor after school for all the classes. Don't quote me on this but I believe I heard that the essay questions were being removed from the SAT to partly address this.
Whereas no one, nowhere, really cares about your ability to answer multiple choice questions quickly.
If they were really revamping the SAT to be an accurate test of students' raw problem-solving skills, rather than their skill at cramming and practicing arbitrary, contrived classes of problems that haven't changed significantly in twenty years, then there would be nothing BUT essays and extended-response questions on the test.
Now you bring up another interesting point. Is the point of school the general teaching of academic knowledge or is it for teaching you to be a well rounded adult (I think it should be for a well rounded adult. Social skills is very important and adult day to day interaction. Would we grade someone who spends all there time lower for lack of day-to-day social skills?
Thing is, if X units of work in a dedicated SAT prep class improves your SAT score by 50 points, while X units of work in a class teaching trigonometry only improves your SAT score by 15 points... does that mean that trigonometry is less important than taking more SAT prep? Or that a student who spent the time on SAT prep is an objectively better student?That said, doing SAT prep classes does require actual work. I'd far rather the opportunity to achieve on this test through hard work that is objectively measured.
[Before you answer, reflect that trigonometry is deemed 'too advanced' to be required on the SAT. And yet basically all colleges expect that any student with a scrap of mathematical or technical aptitude will have taken one or more classes that use it before graduating from high school]
So in short, as an 'objective measure of hard work' the SAT sucks, because even on subjects it purports to measure (math skills), it doesn't measure all high school math skills. It only measures some of them. And even the ones it does measure, it measures in a very specific, stilted way. So there's an incentive to game the system by learning "test-taking strategies" that have nothing to do with how much actual reading, writing, and mathematics you know.[/quote]
It is my opinion that more complex problems should be left for the class room and class room tests. The point of standardized test is an objective general assessment of skills that you take away with you and accumulate over the years, not something that you will forget right after taking the class. The CPA exam for example tests on a large breath of concepts but does not go to deeply into any one like you might takle in a class.
So, basically, a student who self-studies and practices a thousand SAT questions will hopefully get the same benefit as Richard J. Butterworth VI gets from their tutors. See, I believe that... but the problem is that the SAT clearly isn't a fair or objective test.Although requiring more self motivation, self-study is definitely an option, and a much cheaper one. There are prep books at the library and far cheaper self study resources available to purchase. The internet has also tremendously leveled the playing field in the last few years even compared to a decade ago. It is mind blowing easy to google a concept that you don't understand these days and read up and figure it out.
It doesn't matter that in theory, a person who has nobody to show them the tips and tricks for beating the SAT can do as well as a person who has thousand dollar tutors to show them how. The point is that the people who can splurge a thousand for a tutor get advantages over the ones who don't. It's not as level a playing field as you make it out to be.
And if you wanted to make the test more objective you'd have to make the questions more challenging, rigorous, and complicated... which would mean they would NOT look like the old familiar multiple choice tests we're used to in America.[/quote]
There are definitely advantages to a tutor, I don't disagree, but I feel that those advantages are more readily impacted in a day-to-day class room environment were the tutor is able to help on day-to-day tasks. A standardized general assessment test is a pretty personal journey that someone has to take and drill into their head.
How does making a test more challenging and rigorous and complicated make it more objective? Objectivity comes from the fact that it is standardized and you are taking human grader judgement out of the picture.
Sorry ran out of time will get to your other points later. But I will admit I don't have too much background on the standardized public school assessments themselves, I'm just pro standardized test in general.
Brotherhood of the Monkey @( !.! )@
To give anything less than your best is to sacrifice the gift. ~Steve Prefontaine
Aoccdrnig to rscheearch at an Elingsh uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht frist and lsat ltteer are in the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae we do not raed ervey lteter by it slef but the wrod as a wlohe.
To give anything less than your best is to sacrifice the gift. ~Steve Prefontaine
Aoccdrnig to rscheearch at an Elingsh uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht frist and lsat ltteer are in the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae we do not raed ervey lteter by it slef but the wrod as a wlohe.
-
- Emperor's Hand
- Posts: 30165
- Joined: 2009-05-23 07:29pm
Re: Should parents legally pay for their ADULT children's tu
Perhaps. However, if you don't grade people's ability to write essays and present rational, coherent arguments... you really have not measured their intelligence.ArmorPierce wrote:I can be mistakened but I beleive that a lot of it was due to foreign test takers were disadvantaged in taking the Essay and it felt that it favored certain demographic
It's a serious problem- because to assess intelligence and the skills that matter in higher education or the workplace, you have to ask complex questions. Complex questions require complex language. Which means the test has to use complex language... which in turn makes it less accessible to people unfamiliar with the language.
Writing standardized test questions that are linguistically simple but require complex thought to answer is VERY hard. Hard enough that I'm not inclined to bet anything I'd care to lose on the test-writers succeeding.
I cannot comprehend the question at the end of this paragraph. Could you rephrase?Now you bring up another interesting point. Is the point of school the general teaching of academic knowledge or is it for teaching you to be a well rounded adult (I think it should be for a well rounded adult. Social skills is very important and adult day to day interaction. Would we grade someone who spends all there time lower for lack of day-to-day social skills?
Except that this invariably means that such tests lose their ability to accurately model workplace performance. It isn't assessing your ability to use skills in context, to make sense of a complicated problem, to present a multi-step reasoned approach to problem solving.It is my opinion that more complex problems should be left for the class room and class room tests. The point of standardized test is an objective general assessment of skills that you take away with you and accumulate over the years, not something that you will forget right after taking the class. The CPA exam for example tests on a large breath of concepts but does not go to deeply into any one like you might takle in a class.
It also means that the savvy test-taker will prepare for tests by accumulating "broad but shallow" knowledge that makes them much less effective and proficient when the time comes to use that knowledge.
Have you actually had tutoring on these tests? You are not a representative sample of the population, and your personal experience may not leave you well informed of how people with different backgrounds achieve their scores.There are definitely advantages to a tutor, I don't disagree, but I feel that those advantages are more readily impacted in a day-to-day class room environment were the tutor is able to help on day-to-day tasks. A standardized general assessment test is a pretty personal journey that someone has to take and drill into their head.
If the test is not challenging, rigorous, and complicated, it is objective but worthless.How does making a test more challenging and rigorous and complicated make it more objective? Objectivity comes from the fact that it is standardized and you are taking human grader judgement out of the picture.
It's like, imagine a test that consists entirely of one question: "What is your name?"
It's objective because it measures whether people can spell their own name. But it's worthless because all it tells you is that basically everyone can spell their name, including people who are functionally illiterate and write at a second-grade level.
Suppose you then decide to make the test longer: "spell your name 100 times." Now it's nothing but a test of speed penmanship. Again, it's objective- you can measure objectively who can write their name down 100 times fast. And how fast they did it. But the results are meaningless, because who cares?
So to be meaningful the test results have to be challenging, have to reflect something other than "here are the five types of problems, memorize how to do them quickly and you'll ace the test even if you have no understanding of how to apply what you just learned."
Since standardized public school assessments ARE, for all intents and purposes, the single biggest growth area in standardized testing in general... if you don't know anything about them, you don't know much about standardized tests.Sorry ran out of time will get to your other points later. But I will admit I don't have too much background on the standardized public school assessments themselves, I'm just pro standardized test in general.
This space dedicated to Vasily Arkhipov
- ArmorPierce
- Rabid Monkey
- Posts: 5904
- Joined: 2002-07-04 09:54pm
- Location: Born and raised in Brooklyn, unfornately presently in Jersey
Re: Should parents legally pay for their ADULT children's tu
The more complex the questions asked the narrower the focus the test can take typically. SAT scores have a 80% correlation with IQ test score which would be evidence for its use as a measure of intelligence. http://pss.sagepub.com/content/15/6/373Simon_Jester wrote:Perhaps. However, if you don't grade people's ability to write essays and present rational, coherent arguments... you really have not measured their intelligence.ArmorPierce wrote:I can be mistakened but I beleive that a lot of it was due to foreign test takers were disadvantaged in taking the Essay and it felt that it favored certain demographic
It's a serious problem- because to assess intelligence and the skills that matter in higher education or the workplace, you have to ask complex questions. Complex questions require complex language. Which means the test has to use complex language... which in turn makes it less accessible to people unfamiliar with the language.
Writing standardized test questions that are linguistically simple but require complex thought to answer is VERY hard. Hard enough that I'm not inclined to bet anything I'd care to lose on the test-writers succeeding.
I have heard people counter that IQ test itself is not a good measure of intelligence. If that is the case, then what is? What can be used as an objective measure of intelligence? If the answer is nothing and response is elimination of standardized testing then you have to rely human judgement which brings with it all the fun of nepotism, prestige and established network. We know human judgement is flawed, people will push for their own first. Nepotism, racism, and classism are issues that exist today.
Yes, those who come from privileged backgrounds still have the advantage due to increased opportunities and options such as tutoring but I feel like that the alternative is worse.
Okay.I cannot comprehend the question at the end of this paragraph. Could you rephrase?Now you bring up another interesting point. Is the point of school the general teaching of academic knowledge or is it for teaching you to be a well rounded adult (I think it should be for a well rounded adult. Social skills is very important and adult day to day interaction. Would we grade someone who spends all there time lower for lack of day-to-day social skills?
What I am getting at is that, no most jobs will not care how good you are at multiple choice tests. Most jobs will also not care about if you know anything about history, politics, science unless you are going directly to that field. Should that be eliminated?
On the flip side, I would say that 'social skills' and 'communications skills' probably are the most important skills for being successful at your job, assuming you are competent which is quite frankly a low hurdle to surmount. Should schools place a higher emphasis on 'social skills' and 'communication skills.' Should students be graded on this?
To summarize, I think while yes we should consider skills that employers would consider important, I don't think that it should be the only or even the main factor in developing a well rounded academic education. I think that most jobs today don't necessarily need as much education as is being required. Shoot, in my field just a couple decades ago there were people who were able to work their way up from the mail room but it is something you would be hard pressed to find today.
I disagree. I feel that it provides a solid foundation that can be pulled from in order to successfully solve the more complex problems.Except that this invariably means that such tests lose their ability to accurately model workplace performance. It isn't assessing your ability to use skills in context, to make sense of a complicated problem, to present a multi-step reasoned approach to problem solving.It is my opinion that more complex problems should be left for the class room and class room tests. The point of standardized test is an objective general assessment of skills that you take away with you and accumulate over the years, not something that you will forget right after taking the class. The CPA exam for example tests on a large breath of concepts but does not go to deeply into any one like you might takle in a class.
It also means that the savvy test-taker will prepare for tests by accumulating "broad but shallow" knowledge that makes them much less effective and proficient when the time comes to use that knowledge.
No I have never had a tutor on these tests. I have tutored others and am basing my opinion on my personal experience.Have you actually had tutoring on these tests? You are not a representative sample of the population, and your personal experience may not leave you well informed of how people with different backgrounds achieve their scores.There are definitely advantages to a tutor, I don't disagree, but I feel that those advantages are more readily impacted in a day-to-day class room environment were the tutor is able to help on day-to-day tasks. A standardized general assessment test is a pretty personal journey that someone has to take and drill into their head.
If it was as easy as that then what is the problem? Why are so many people having problems with the standardized tests?If the test is not challenging, rigorous, and complicated, it is objective but worthless.How does making a test more challenging and rigorous and complicated make it more objective? Objectivity comes from the fact that it is standardized and you are taking human grader judgement out of the picture.
It's like, imagine a test that consists entirely of one question: "What is your name?"
It's objective because it measures whether people can spell their own name. But it's worthless because all it tells you is that basically everyone can spell their name, including people who are functionally illiterate and write at a second-grade level.
Suppose you then decide to make the test longer: "spell your name 100 times." Now it's nothing but a test of speed penmanship. Again, it's objective- you can measure objectively who can write their name down 100 times fast. And how fast they did it. But the results are meaningless, because who cares?
So to be meaningful the test results have to be challenging, have to reflect something other than "here are the five types of problems, memorize how to do them quickly and you'll ace the test even if you have no understanding of how to apply what you just learned."
Brotherhood of the Monkey @( !.! )@
To give anything less than your best is to sacrifice the gift. ~Steve Prefontaine
Aoccdrnig to rscheearch at an Elingsh uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht frist and lsat ltteer are in the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae we do not raed ervey lteter by it slef but the wrod as a wlohe.
To give anything less than your best is to sacrifice the gift. ~Steve Prefontaine
Aoccdrnig to rscheearch at an Elingsh uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht frist and lsat ltteer are in the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae we do not raed ervey lteter by it slef but the wrod as a wlohe.
-
- Emperor's Hand
- Posts: 30165
- Joined: 2009-05-23 07:29pm
Re: Should parents legally pay for their ADULT children's tu
The answer is that the creation of standardized tests is not yet a mature art.ArmorPierce wrote:The more complex the questions asked the narrower the focus the test can take typically. SAT scores have a 80% correlation with IQ test score which would be evidence for its use as a measure of intelligence. http://pss.sagepub.com/content/15/6/373
I have heard people counter that IQ test itself is not a good measure of intelligence. If that is the case, then what is? What can be used as an objective measure of intelligence? If the answer is nothing and response is elimination of standardized testing then you have to rely human judgement which brings with it all the fun of nepotism, prestige and established network. We know human judgement is flawed, people will push for their own first. Nepotism, racism, and classism are issues that exist today.
This is the idea I'm struggling to get across to you- that the 'technology' of creating tests that accurately measure what they purport to measure is still under development. Therefore, while any given standardized test is clearly an objective measure of something, we can't be sure it objectively measures what we think it does.
So we can't just... worship the test scores and assume they tell us everything we need to know. It's not the equivalent of holding a ruler up to someone to measure, clearly and precisely, their height. Where we know exactly what we just measured, and how, and why we care, and how to compensate for any biases in the measuring equipment.
It's more like pointing a black box at someone, pushing a button, and getting a number. When you ask what that number means, you get the reply "It means, uh... something. And bigger numbers are better. We're pretty sure."
So, use the tool with caution because it may not be doing what you think it does. And, applied excessively or carelessly, it may have bad side effects.
My argument is not all about the workplace. It's also about higher learning- to get a meaningful college education you have to have a wider range of thinking skills. And, though I didn't mention it before, it's also about your ability to function as a citizen in a democracy, as an informed participant in a market economy. Doing these things requires people who can think, who understand context and can take a spray of random facts and assemble them into a coherent picture of the world.What I am getting at is that, no most jobs will not care how good you are at multiple choice tests. Most jobs will also not care about if you know anything about history, politics, science unless you are going directly to that field. Should that be eliminated?
(And, incidentally, that kind of thing is where knowledge of history, politics, and science helps)
Thing is... these higher level skills are worth something outside high school. Multiple choice problems aren't.
While in certain times and places it probably WOULD make sense to have classes devoted entirely to teaching behavioral norms (and thinking skills), in general the entire society around a child does a better job of teaching them to socialize than a school itself ever could.On the flip side, I would say that 'social skills' and 'communications skills' probably are the most important skills for being successful at your job, assuming you are competent which is quite frankly a low hurdle to surmount. Should schools place a higher emphasis on 'social skills' and 'communication skills.' Should students be graded on this?
By contrast, the school must teach algebra; the environment usually won't do it for them.
Again, though, the "well rounded education" means you know a lot of things... but it does NOT mean you're good at sorting through multiple choice questions and bubbling in correct answers in an average of 90-120 seconds per problem for two hours at a stretch. When assessing whether someone actually has a well rounded education, standardized tests as we now know them only take us so far.To summarize, I think while yes we should consider skills that employers would consider important, I don't think that it should be the only or even the main factor in developing a well rounded academic education. I think that most jobs today don't necessarily need as much education as is being required. Shoot, in my field just a couple decades ago there were people who were able to work their way up from the mail room but it is something you would be hard pressed to find today.
I disagree. I feel that it provides a solid foundation that can be pulled from in order to successfully solve the more complex problems.Except that this invariably means that such tests lose their ability to accurately model workplace performance. It isn't assessing your ability to use skills in context, to make sense of a complicated problem, to present a multi-step reasoned approach to problem solving.It is my opinion that more complex problems should be left for the class room and class room tests. The point of standardized test is an objective general assessment of skills that you take away with you and accumulate over the years, not something that you will forget right after taking the class. The CPA exam for example tests on a large breath of concepts but does not go to deeply into any one like you might takle in a class.
It also means that the savvy test-taker will prepare for tests by accumulating "broad but shallow" knowledge that makes them much less effective and proficient when the time comes to use that knowledge.
No I have never had a tutor on these tests. I have tutored others and am basing my opinion on my personal experience.Have you actually had tutoring on these tests? You are not a representative sample of the population, and your personal experience may not leave you well informed of how people with different backgrounds achieve their scores.There are definitely advantages to a tutor, I don't disagree, but I feel that those advantages are more readily impacted in a day-to-day class room environment were the tutor is able to help on day-to-day tasks. A standardized general assessment test is a pretty personal journey that someone has to take and drill into their head.
If it was as easy as that then what is the problem? Why are so many people having problems with the standardized tests?[/quote]If the test is not challenging, rigorous, and complicated, it is objective but worthless.How does making a test more challenging and rigorous and complicated make it more objective? Objectivity comes from the fact that it is standardized and you are taking human grader judgement out of the picture.
It's like, imagine a test that consists entirely of one question: "What is your name?"
It's objective because it measures whether people can spell their own name. But it's worthless because all it tells you is that basically everyone can spell their name, including people who are functionally illiterate and write at a second-grade level.
Suppose you then decide to make the test longer: "spell your name 100 times." Now it's nothing but a test of speed penmanship. Again, it's objective- you can measure objectively who can write their name down 100 times fast. And how fast they did it. But the results are meaningless, because who cares?
So to be meaningful the test results have to be challenging, have to reflect something other than "here are the five types of problems, memorize how to do them quickly and you'll ace the test even if you have no understanding of how to apply what you just learned."
This space dedicated to Vasily Arkhipov
Re: Should parents legally pay for their ADULT children's tu
Yeah, I agree. But there doesn't seem to be a known alternative since grading multiple choice problems is efficient, and can even be done by a machine. It might not be practical to grade thousands of short-answer questions - plus multiple choice questions are objective; there's definitely only one right answer, whereas short-answer questions may possibly fall under the disgression of the subjective judgment of an individual grader, which introduces another variable into the test scores.Simon Jester wrote:Thing is... these higher level skills are worth something outside high school. Multiple choice problems aren't.
Perhaps it's better to eliminate school-wide standardized tests entirely, and simply rely on individual GPA.
-
- Emperor's Hand
- Posts: 30165
- Joined: 2009-05-23 07:29pm
Re: Should parents legally pay for their ADULT children's tu
Some progressive and interesting options associated with the new Common Core tests:
1) Questions in which you are given, say, ten possible answers, and asked to identify all the correct ones, or all the ones that are incorrect for a certain reason, or something of that nature. This tests the same skill set as multiple choice, but is more open-ended. There is more than one correct choice, so you can't just stop at eliminating one or two answers and making a kinda-educated guess.
2) Questions in which the answer is open-ended but there is a defined rubric. This cannot be scored by machine but can be made pretty objective; it's how the AP tests work.
3) In general, any question in which the problem prompt doesn't come with canned answers: "highlight the relevant passage of the text that supports claim XYZ," for instance.
1) Questions in which you are given, say, ten possible answers, and asked to identify all the correct ones, or all the ones that are incorrect for a certain reason, or something of that nature. This tests the same skill set as multiple choice, but is more open-ended. There is more than one correct choice, so you can't just stop at eliminating one or two answers and making a kinda-educated guess.
2) Questions in which the answer is open-ended but there is a defined rubric. This cannot be scored by machine but can be made pretty objective; it's how the AP tests work.
3) In general, any question in which the problem prompt doesn't come with canned answers: "highlight the relevant passage of the text that supports claim XYZ," for instance.
This space dedicated to Vasily Arkhipov