Ask-Dr-Kirk

Time Off For Good Behavior

I'm off to Belgium and the Netherlands for ten days so will be taking a short break from posting on this website.  Good luck everyone in wrapping up your semester!

Determining The Validity And Reliability Of Rubrics

Dear Dr. Kirk:

I have a question about using a rubric...We have several tasks that we would like students in different programs to do. The tasks would be the same in that the same kind of questions would be asked and the same kind of projects would be done. However, they would be specific to each program (for example - early childhood activities look different from elementary children activities).  If we use a task-specific rubric, would the results give us valid score-based inferences?

Sincerely, JY

Dear JY:

You might check out this article by Barbara M. Moskal and Jon A. Leydens (Colorado School of Mines). As they note:

Although many teachers have been exposed to the statistical definitions of the terms "validity" and "reliability" in teacher preparation courses, these courses often do not discuss how these concepts are related to classroom practices...One purpose of this article is to provide clear definitions of the terms "validity" and "reliability" and illustrate these definitions through examples. A second purpose is to clarify how these issues may be addressed in the development of scoring rubrics. Scoring rubrics are descriptive scoring schemes that are developed by teachers or other evaluators to guide the analysis of the products and/or processes of students' efforts.  The ideas presented here are applicable for anyone using scoring rubrics in the classroom, regardless of the discipline or grade level.

Does anyone else have suggestions or resources on determining the validity of a rubric?

Tip To See If Students Are Really Listening In Class

Angela Maiers offers the following tip to see if students are really listening in class especially if they are not "actively" participating by speaking up.  She calls it the TWO WORD STRATEGY.

"Here is how it works:  Choose a point in the lesson or group discussion that you think warrants reflection or active learning. Ask the students or group members to stop, reflect on the content presented, and synthesize their most important thoughts and ideas down to only two words. This may sound easy, but it is difficult to express yourself in only two words. Be sure to encourage, not judge as learners share out their responses."

As she notes, this strategy:

• can be used with any audience: regardless of any age, grade, or content.
• can be employed at any time: before the discussion starts, during the presentation, or afterwards as participants reflect on key ideas.
• is a powerful assessment tool: In an instant, I can hear what my audience is thinking and wondering based on their two words shared.
• allows everyone to participate: the two words can be written down, shared with a neighbor, or communicated as a  group. Everyone has a chance to pause and reflect about what matters most to them as active participants in the learning experience.

Studying For Final Exams

I went down to the coffee shop on campus Monday morning and found at least half of my class studying for the final to be given later that day.  This isn't something I typically think to suggest but I think it's a great idea for students to study in groups...they learn from each other as well as from me!

Advice For Professors: What They Didn't Teach You in Graduate School

Professors Paul Gray and David E. Drew offer advice from their new book, What They Didn't Teach You in Graduate School: 199 Helpful Hints for Success in Your Academic Career (Stylus Publishing, 2008). As they note:

“Most new Ph.D.'s who accept faculty positions are shocked to discover that no one told them what their day-to-day jobs would really entail. They struggled as graduate students to master the literature, theories, models, and analytical techniques in their fields and wrote dissertations of which they are proud, but they quickly realize that this knowledge is separate and distinct from understanding and dealing with the challenges and obstacles that face a beginning professor.”

Some tidbits from the book: 

Finish your Ph.D. as quickly as possible. Don't feel that you need to create the greatest work that Western civilization ever saw. Five years from now the only thing that will matter is whether you finished.

Don't take a tenure-track faculty position without the Ph.D. in hand. We estimate the odds are two to one against your ever finishing your degree. Furthermore, without a Ph.D. you will be offered a significantly lower salary, and you may never make up the difference.

Know that publications are your only form of portable wealth. Prioritize accordingly.

Never, ever choose sides in department politics. The side you are on expects your support and will give you no reward for it. The side(s) you are not on will remember forever.

Never become a department chair unless you are already a tenured full professor. Yes, it will reduce your teaching load. Yes, it will give you visibility. No, it will not confer power on you.  Most department chairs do less research and publish less while in that position than they would as a faculty member. Thus you are producing less portable wealth per year, and you are reducing your chances for tenure or for promotion.

Write most of your articles for refereed journals. Papers presented at meetings get you funds to be a world traveler. However, even if refereed, conference papers don't really count for tenure, promotion, or salary raises.

Do, however, serve as a reviewer for journals, particularly top journals. Treat this job seriously. You will see much junk being submitted and appreciate why some journals reject 80 percent or more of their submissions. You will develop an aesthetic for what is good and what is not. You will correspond with some powerful people. When you do get a good paper to review, you will receive much earlier knowledge of an important new development. And the information gained is worth more than the time you take reviewing.

I've been teaching at the college level for 26 years and would agree with their advice.  Unfortunately, some of this I had to learn the hard way my first few years.

How Was Stonehenge Built?

This is interesting.  Wally Wallington demonstrates how he thinks Stonehenge was built.

The Frustration And Satisfaction Of Learning Something New

Yesterday I met with Angela Maiers and Mike Sansone while I was visiting in Des Moines (Thanks Mike for the great lunch at Huhot!).  They decided that as it was the second anniversary of the date I started my teaching blog that I was ready to learn something new.  I am now on Twitter and soon hope to develop my own wiki for my classes.

Professor Lisa M. Lane (MiraCosta College) writes about the fear of learning new things...when everyone else seems to be ahead of you and you're struggling to figure things out.

"I dreamt the other night that I went to the first day of class as a student, an upper division or graduate class of some kind. About 20 students. The professor walks in and begins an interactive lecture. I realize I don’t understand what he’s talking about. I look around and see that each of the other students has a stack of about 7 books. Seeing I’m lost, a helpful student opens one and points to a chapter for me. But it’s obvious from the way the professor is talking that everyone has read this already. Before the first day of class. Everyone is quite literally “on the same page”, and I have no idea what’s being discussed."

I think it's good for us as teachers to attempt something new and thus remember the frustration and then the satisfaction of learning, something our students experience on a regular basis.

It's My Anniversary!

Two years ago I started sharing my teaching tips as well as those of others on this site.  Thanks to Tim Johnson and Mike Wagner for encouraging me to become a blogger and to Mike Sansone for teaching me the "how to" skills.  I also want to thank all the readers who have commented on the blog or emailed me with their own tips, stories, and links.  I look forward to another great year of teaching and learning!

Can We Give Out Information On Our Students?

In an article published this week in the Chronicle of Higher Education, Steven J. McDonald clarifies the privacy act, Ferpa, that affects the information we give out about our students. Interesting points include:

While Ferpa applies to official records (such as transcripts), it does not apply to personal knowledge.  “Thus, for example, a professor who observes a student behaving oddly in a classroom, a resident assistant who notices a disturbing change in a student's temperament, or an adviser who sees a student become increasingly withdrawn and uncommunicative is free, as far as Ferpa is concerned, to raise the concern with others — and should do so. We do neither the student nor ourselves a favor if we don't try to reach out and deal with such situations when we still have the opportunity.”

In addition, McDonald addresses the issue of giving out information to parents: 

"Primary control over a student's records does shift from the parents to the student when the student enrolls in college, even if the student is still a minor, but primary control is not the same as total control. Institutions can disclose student information to parents under any number of circumstances:

• If either parent claims the student as a federal tax dependent, the institution may, with confirmation of that status, disclose any and all information it has about the student to both parents, regardless of the student's age or whether there is an emergency.
• If the student is under 21, the institution may inform the student's parents of any violations of its alcohol or drug policies, regardless of whether the student is a tax dependent or whether there is an emergency.
• If the institution reasonably believes that there is a health or safety emergency involving the student, the institution may alert the student's parents and seek their assistance, regardless of the student's age or whether the student is a tax dependent.

Moreover, we can make such disclosures even if the student has asked us not to. Ferpa doesn't give students a veto over any of the permitted disclosures except the one for "directory information."

Certainly if a student told me about personal issues that he/she is struggling with or if I observe a student is having trouble coping with stress, I would suggest to the student to see a school counselor.  I've even made the call myself and walked the student to the counselor from my office.  However, in this day of "helicopter parents," I find it easier to refuse to answer any questions from parents about their child's grades or attendance and to refer the parent to an administrator.

Repeat After Me...Would You Like Fries With That?

Ben Stein writes about an interesting experience he had recently at a Barnes and Noble store where the employees were college students. 

As he and his friend walked back to his apartment, he said,

"That was amazing. Those people didn't know how to do a basic transaction like mailing books."

His friend corrected him:  "No...they don't want to do it. They didn't even really try very hard. They're college students. That means they don't care at all. They're getting paid whether they send the books or not. They're like civil servants -- only civil servants have a good attitude, and these guys have a poor attitude."

Unfortunately, I have seen this same attitude with some of my own students.  Many of my students are hard working, attending school while working 30-50 hours a week.  They seem to understand the importance of education to their lives and careers.  However, others (and more others than I would like to see) feel it is a chore to attend classes and do the work required.  They are indignant that I expect them to be on time.  And they make decisions which I don't understand.  Recently I announced that at the end of the three hour class, I would be giving them an opportunty to earn additional points on an in-class assignment as they had not done well on their exam.  Several of the students did not bother to stay until the end, instead choosing to slip out during the break without saying anything to me.  These students, of course, are the very ones who could have really used the points.