Skip to Navigation | Skip to Content
Faculty Resource Network Home

Approaching Assessment: the Road Ahead

November 21-22. 2003

Each year, the federal government provides more than $60 billion in grants and loans for students via, financial aid programs that are governed by the provisions of the Higher Education Act of 1965. With the bill reauthorizing the Higher Education Act presented to Congress in 2004, the topic of assessment promises to become one of the key issues confronting institutions of higher learning and their faculty.

In the fall of 2003, faculty members and administrators from member institutions of the Faculty Resource Network and Leadership Alliance convened at New York University to discuss the implications of this critically important topic.

The debate surrounding the reauthorization of the Higher Education Act raises questions about the ways in which federal funding can be quantified into verifiable outcomes.

  • How well do students perform?
  • How do we monitor and assess both student and faculty performance?
  • What kinds of retention and graduation rates can institutions point to?
  • Are there common standards of performance? Should there be? If so, who is responsible for setting them?
  • How should institutions assess their own performance?

Following are excerpts from the two plenary sessions of the symposium:

Keynote Conversation

The Keynote Conversation at the Assessment Symposium was convened by Sharon Weinberg, Vice Provost for Faculty Affairs at New York University, and featured the following panelists:

Catharine Stimpson, Dean, Graduate School of Arts and Science, New York University; Jane Oates, Senior Education Adviser to U.S. Senator Edward Kennedy (D-MA); and Alicia Hurley, Director, Office of Federal Policy, New York University.

The following excerpts pick up mid-conversation regarding the reauthorization of the Higher Education Act:

Oates: Iíll speak for Senator Kennedy and not for all Senate Democratsóalthough many of them would agree with meóthat access will continue to be the major priority in this reauthorization. Weíre going to expand the definition of access to mean persistence and graduation rates. In order to do that we have to have clearer ideas, first of all, about whatís happening to people when they enter college, disaggregate it as much as we can by gender, race, and socioeconomic class, and look at what happens to them. We do have a program that looks at developing partnerships between two- and four-year institutions. We know that the point of entry for the most challenged students is a two-year community college, and unless we have better ability to have a seamless transition for students from a two-year to a four-year institution, weíre not going to get those students to bachelorís degrees. Right now, nationwide, about nine percent of the people who start at a community college with the hope to going on to a bachelorís degree actually move to a four-year college. For Senator Kennedy, true success in access is when academically qualified students attend college, persist in college, and retain degrees at the same level whether their parents make $75,000 a year or less than $25,000 a year. Thatís where we are, thatís where weíll measure success.

We are appalled in our office that two equally talented kids have very different outcomes. About 85 percent of academically talented kids from families above the $75,000 annual income level attend a four-year college. Fewer than 45 percent of low-income kids with the same academic talent go to a four-year college. Our country canít survive if that inequity persists. Thatís the basis of our Senate bill. It strengthens TRIO and Gear Up. It looks at this new program, which will require, if passed, all institutions to disaggregate their data based on early decision, what happens with kids who go in through early decision, and what happens with regular admission, which is going to be difficult. But we had to make a stand on early decision. If students have a better chance of persisting if they enter through early decision, or if they have a better chance of getting accepted in certain institutions if they choose early decision, we need to get that information out to first-generation families so all students can take their SATís early in their junior year, and so that all students can be prepared to have their applications done by the 15th of November instead of the 15th of January.

Thereís no hidden agenda here, thereís no sanction intended. Nobodyís trying to get rid of early decision. All we are trying to do is level the playing field so that all students, even if they go to a mediocre school system, get the information that if they want to go to the University of North Carolina at Chapel Hill, they have a better chance of getting in if they do early decision. In some schools like my alma mater, Boston College, the numbers arenít that way. You have a better chance if you wait to the general pool, but all families and all counselors have a right to know that. So thatís one take on how we improve the process a little.

Hurley: I think there was concern about that particular provision. I myself wondered if it might not have the opposite affect; that is, if minority students or low-income people who see these numbers would automatically not apply for early decision based on the data. And I wondered if we are trying to get at this idea of getting students into college, but not doing a good job of putting out a lot of data. How do you get all this data and make it meaningful. So now we would disaggregate the information and put it out, but launching it into some database doesnít mean that people still wonít go to the US News and World Report and look at that instead of the United States Department of Education web site.

Stimpson: So weíre talking now about two forms of access, each of which could be an accountability measure. One is access to institutions through graduation. The other is access to data that could help you make an informed decision whether youíre a policy maker, a parent, or a student. Youíre saying in one instance that access to the institutions is not blocked and youíre saying access to the data is blocked because of the indigestible format.

Oates: Iíve spent the bulk of my career in the Boston and Philadelphia public schools. While I donít know what it is like to be poor, I do know a lot about poor people and poor families, and I can tell you honestly they are not buying US News and World Report. If we want to get information to poor families, weíve got to get it to their schools. Maybe weíll get to a point, if we ever close the digital divide, that putting it on the net will be enough, but most of the families that I worked with didnít own a computer and went to schools where I taught. If they were lucky, they could get their hands on a computer after school for an hour a week.

Stimpson: Is there anything in the fine print of the Republican or the Democratic or the House or the Senate versions that makes access to data better for poor people before school?

Hurley: So far what you see is just more of it. They want to put more data out there, and thatís not a bad proposition. Itís just whether or not they can figure our how to best deliver it. There are better ways to get information to the public than putting out databases that just show raw data, which is sort of where they are now.

Stimpson: Youíve mentioned that we are moving into an election year. When this Congress is over, what role do you see the White House playing in the reauthorization, and how do either of you see presidential politics playing itself out, or does that depend on who the Democratic nominee is? If you had to look into the future, how would you see this working out?

Hurley: I hope the White House doesnít take it up as an issue, because I think that would force this to become even more partisan than it already is. It has not traditionally been the case that we go into reauthorization with a Republican set of issues and a Democratic set of issues laid out so clearly. So I think if this White House gets involved in the conversation, we are in for a lot of trouble.

Oates: I agree. They have told us that they are not going to come up with the bill, but since January of 2001 this administration has not written one piece of legislation. Itís unheard of because Bush Senior, sent up legislation, and as we know the Clinton administration sent up legislation. I also hope they donít get involved in it, because ifthis President and this Secretary of Education get into it, weíre going to have the higher education version of ďNo Child Left Behind.Ē They are already saying the same buzz wordsóaccountability and flexibilityóand every time a Republican says that on a panel with me about institutional accountability, I say itís a market-driven system: get your talking points from your commerce staffer.

Republicans, I thought, believe in free trade. In higher education if a student isnít happy they are going to go, and thatís how institutions can be held accountable. I think if we see the President get involved in it, I think weíll see their hand in the State of the Union message. It will be very interesting to see what they do on Pell Grants. Since Pell is an entitlement it goes through trends, years where thereís a surplus and years when thereís a shortfall. Because of the economy, in January the budget office is now predicting that if Medicare passes then we will have close to a trillion-dollar deficit. That is going to put us in a very different place, and they keep talking about this Pell shortfall. They have not given us quarterly numbers since April, but I assume we are close to a $3 billion shortfall. But thatís because the economy is in terrible straits, and people are choosing to go back to school. Donít forget that Pell is about equally used between traditional 18 to 22 year olds and adult learners. So disadvantaged adults go for their training using Pell Grants. It will be interesting to see if the President comes out with an increase in Pell Grants or whether he just fulfills the Pell shortfall in his budget. We have not given the Secretary of Education the authority since January of 2001 to adjust the base grants. Senate Republicans have stood firm with us on that. Prior to this the Secretary did have the ability to reduce the maximum grant annually and we have not given the Secretary that authority. I hope we maintain that in this Labor-Education Spending bill.

Stimpson: If ďNo Child Left BehindĒ was really extended to higher education, what would it look like?

Hurley:I think we would have great test scores, probably put some mandatory tests into the system.

Stimpson:The whole, public and private?

Hurley: Amazing, isnít it? This is what they use; they use Title IV Funding to do things to private and public. What would a worst-case scenario look like? You would start having real performance measures for all students regardless of where they were going. Jane, what else?

Oates: Teachers wouldnít be the only victims in higher education. I think everybody would have an exit test if the extreme happened. You might even have annual tests to make sure that college students were making progress. You talk about the worst-case scenario, I think that this aggregation stuff will happen in a number of scenarios any way, but I do think that testing is something that is not out of the realm.

Stimpson: Undergraduates only?

Oates: I think so. I think that the House does not fully grasp graduate school. But the other piece that we havenít really talked about is accreditation. Mr. Petri, who is normally a very reasonable Republican member of the House committee, has gone the "Lynn" Cheney route and said that he wants to get rid of all accrediting. If we get rid of accrediting, what would then be the condition for Title IV participation? If accrediting goes it leaves a huge void, a void that would be filled by many people who are career staffers and many members who did not go to a four-year institution or werenít happy when they went there. I think it puts a huge pressure on what they would do instead of accrediting.

Stimpson: So people here in this room who have worked for accrediting agencies, like Middle States, might be looking for work?

Hurley: I donít think you have to worry about losing your jobs, but itís interesting that they would even raise it. Why would you go into something and say we want lless of accreditation but more accountability? Where is the theory in this? They had to make these changes in the early 90s after all the high default rates on student loans. What we are seeing right now is the House trying to put all of those measures back in to weaken the system at a point when they are really coming down on us on accountability the most.

Stimpson: But this is interesting, because when Mrs. Cheney was actually a government official she and her organization started a whole other accrediting organization. So this is a reversal of directions. You both seem to agree, worst-case scenario, but what are the chances of even a modified version of worst-case scenario?

Hurley: I donít think it can happen. And I hope that the lobbying industry for higher education world is better.

Oates: It will make for a lot of conversation. There is something Alicia said earlier, that when people donít want to put money up they end up having these regulatory and philosophical arguments. But I do think once something is put on the tableócosts, for instanceóit is going to be on the table throughout the debate. The same would be true of accrediting or testing. And Kate, you bring up the amazing bipolar nature of some of our friends inside the beltway. Remember these are the people that hated the National Board for Professional Teaching Standards. Now they love Checker Finn. They are endorsing his assessment before it even exists. So to think that any logic will prevail . . . .

I hate to give homework, but I warned you I was a middle school teacher. Members of the higher education community have never thought that it was necessary to educate elected officials. Theyíve never voted on their issues as a block, and theyíve never really gone in to talk about their issues. Somehow, Iíve gotten them to talk to me in Massachusetts, but mainly because I speak their language. But the issue is this: there has never been a more critical time for you to talk to your elected officials. It is the time for all levels of the higher education community to get engaged in it, because you are the people who understand the nuts and bolts. When we go back to talk about intellectual diversity, you know what that means in a classroom. You understand that. But if people donít understand that itís critically important, they may sell it off for something that they understand. Staff speaks fluent Pell Grant. They speak semi-fluent campus-based aide.

Stimpson: Are you a Cassandra or a Pollyanna about whatís going to happen?

Hurley: I donít know yet. I think Iím a Cassandra.

Oates: Iím definitely a Pollyanna. We just finished the summer on the Work Force Investment Act. Last March somebody bet me dinner at the restaurant of my choice that I was full of manure that we couldnít get that bill finished this year, and that mare is going to take me to a very expensive restaurant. It was a bipartisan process; we got a bill through without any faith-based language and without block grants. I have no faith in the House of Representatives. I have to say I think it brings out the worst in both our parties when one party controls the House, the Senate, and the White House. I think we all find our lowest level there, but I have great faith in the Senate. I think the Senate will prevail on this, and if the Senate does a bipartisan bill it sticks together in conference. These guys are lazy. They donít know substance. The Senate can out-substance the House seven days a week.

Hurley: I think it just makes the House dangerous.

Question: Iím Jean Morris, executive director of Middle States, and since you all rely on us for accountability I did want to say a couple of words. The accrediters saw this coming three or four years ago, and weíve all changed our standards. Iím now talking about institutional accreditors. For example, we accredit all of NYU, not just the specialized programs, and we all have new standards around the country that really focus on student learning. Thatís our answer to this, that it shouldnít be about graduation rates, although they may be relative for other purposes, it should be about whether students are learningóor else we are going to get some of these convenient kinds of measures that were talked about. In terms of educating the legislators, for the first time the regional accreditors, who are again the institutional accreditors, got together and hired a lobbyist. We could barely afford it. Iím really getting an education on whatís involved here. And we created, as a suggestion of this lobbying group, a set of proposals to get around problem of if they get hysterical at the last minute and put it in the wrong place of the bill. So weíve tried to address issues that we think are of concern to show that the educational community really does care about this and wants to meet everybody at least half way, but in a way that makes sense so that it really doesnít get screwed up accidentally. For example, we are supporting open access and transferability credits. We will be willing to do accreditation reviews. Weíve tried to address some of the distance learning issues. Iíd just like to note that in our new standards we do talk about efficiency as well as effectiveness, and all the standards came from our members. There are new ways for internationalization, and we have a strong standard on integrity. We are trying to support the system, which has always allowed different kinds of institutions all to be excellent.

Question: Programs were mentioned that some of us were not familiar with, TRIO and Gear Up. Could you just tell us what those are?

Oates: TRIO is a program that has existed since President Johnsonís war on poverty. If I used its individual program names you would recognize itóTalent Search, Upward Bound, all our access programs. Some work with middle school and high school students. Many more, like Student Support Services, work with students while they are engaged in a two-year or four-year institution. Higher education institutionsócommunity colleges and four-year collegesó apply for them and get the programs for four or five years. I canít tell you the New York numbers, but there are 48 programs in Massachusetts. They are in every state, but we have a heavier concentration in New Jersey, New York and New England than many of the western states because we have more colleges and more students.

Gear Up is a brand new program, from the 1998 reauthorization. It is a five-year program mandated to begin in seventh grade, a K-16 partnership. States can have them or institutions and local partnerships can have them. We donít have a lot of data about it, because if you do the numbers in 1998 for most of those programs the students are just juniors in high school this year. The hope is that it will improve readiness for college, and will increase the number of students who choose to go on to post-secondary education. And itís a very cost-effective program.

Question: My theory is that they will use graduation rates as one of the primary assessment tools in determining whether or not an institution is meeting the expectations of the government. Has there been any conversation about changing the way in which graduation rates are counted so that institutions are not penalized for students who actually transfer to other institutions? Right now you canít count those students in your graduation rate knowing that they have transferred to another institution. Has anyone considered changing that?

Oates: There is a lot of conversation going on in two areas. First, how do you calculate graduation rates at a community college? When you deal with community college populations, students come for a variety of different reasons. Some come to get a very discrete skill sense so that they can get a job immediately, others for recreational kinds of things, and still other for basic adult education. So if you were to use as the denominator in graduate completion rate everybody who entered a community college, you may as well take away all the funding for community colleges today and not even bother doing the numbers.

But the second point that you bring up is one that we are looking at carefully. The I-PEDís data, which is the most used data source of Post Secondary Education information, do track only first time/full time attenders. Everyone in this room would probably agree that the percentage of first-time/full-time attenders is probably greater at NYU than it is at Nassau Community College. Many people come back, but it doesnít mean that somebody who drops out for a while because they have to support their family or something else and comes back, isnít as successful. Right now we only track those completers within 150 percent of the time, or six years for a four-year degree. Thatís not going to cut it when youíre talking about somebody who is working full time. So weíre trying to work with the Department of Education, where the assistant secretary, Sally Stroupe, is the first friend I made in Washington and a really reasonable person. We are trying to figure out a way to improve that data collection so that we can more accurately report on people. Until we have the confidence that that has been corrected, I donít think that we would agree to use graduation rates as a measure, because we see it as completely flawed at this point.

Question: Suppose I really want to write to my Senator or my Representative. I would identify myself as a faculty member. What would I say to get his or her attention, what things do you think would be most important for me to stress as a voter?

Oates: I vote on this issue, and every one of my friends thatís a faculty member or is engaged in higher education plans to vote on this issue. Itís a wonder anybody ever hired me because I was an activist for a long time. I would begin every letter saying, ďIím a constituent who votes in every election, and you need to understand that I am going to track what you are doing on this issue. Iím going to hold you accountable with my vote, and I will work on election day to let everyone like minded know that you either supported these issues or failed to support them.Ē

Hurley: The other thing that you can consider doing is contacting any one of the associations in Washington that have a set of talking points on their web-sites. You can call about student aid, you can call about intellectual diversity. The American Council on Education has covered just about every topic, but if you wanted to get down into the substance, that is actually what the associations do well. Unfortunately, some of them will say one thing, and others will say another thing.

Oates: I encourage you to take the extra few minutes that it requires to look at all the associations. I mean no disrespect to Middle States or any other, but Iím telling you from an office that answers every piece of mail that we get, we know a form letter, and if you send a form letter youíre going to get a form letter back. If you donít send a form letter, and a staff person sees something that looks different and innovative and that is aligned to their state or congressional district, youíre going to get a call.

Question: I have a question about historical context. It seems to me on the Republican side of the House we hear about the need for college cost controls, price controls, and other words that I havenít heard since the days of Nixon. On the Democratic side of the House I hear an almost Libertarian defense of Title Six programs and the market-driven nature of the system. Can you explain to me what has happened here in this role reversal?

Oates: Going back to my first campaigns for Eugene McCarthy and George McGovern, I donít think the Democrats have changed focus very much. They have always been aligned with equal access and level playing fields. When you hear me say things like ďmarket driven,Ē Iím trying to talk Republican because I want a bipartisan bill. I cannot explain to you why the House Republicans have gone to war with certain issues. I donít know if it is just to get attention. Elected officials like to see their names spelled correctly in print. If a press release about college costs gets 45 hits in papers across the nation, that topic is likely to be used again by people who want to get their name in print. The majority party is anxious to do something to get attention without putting any money behind it. The minority party has been less concerned with spending money when itís for domestic causes, and I think that youíre going to see them talk about things like the need to restore the balance between need-based aide, and between grants and loans. The way that the loan industry has grown over the last 15 years is mind numbing. The money that is spent in federal subsidies to lenders so that they donít have to assume any risksówhich I totally understand so that all kids from all incomes get the same loan rateóis appalling. If we could put that same amount of money into need-based aide, we could change the face of this country in four years. Iím trying to become bilingual. I speak fluent Democrat, and Iím trying to learn a little Republican so that I can get them to listen to my side of the argument.

Hurley: If you look at the bills that have come out, I would observeóand Jane will not be happy when I say thisóthat there are some themes that are consistent across both parties, including college costs and accountability. The language right now is about how our institutions, first and foremost, can hold down their costs. And then theyíre willing to get down into the substance. If I was a layperson glancing at the bills, what I would see is both Republican and Democratic initiatives for teacher training, for instanceóincentives to both the programs and the rules. So let me give you kind of the big picture.

Both the Democrats and the Republicans have bills out right now for incentive programs to minority-serving institution programs. But when you get deeper into it you start to see some of the differences, which is what Jane was identifying earlieróhow they are introducing some of the old proprietary and for-profit schools back into the system. But the Democrats arenít holding us completely harmless. They are not being as liberal as one would assume, and for a while we were wondering where they were. The higher education community had to defend itself against the Republicans, and so it wasnít until October that the Democrats emerged. So weíre in the middle of kind of a matchóitís not as Liberal and Conservative. Weíre just at a point where weíve got both ends of the spectrum filling in and weíre going to end somewhere in the middleóhopefully where we get more funding into the system and fewer rules.