Origins, Part III: Demonstrating Quality in Distance Learning

This post originally appeared in my short-lived blog Truth and Beauty, and was moved to this blog in June, 2007.

Part of what's been going on in my professional life that has led me to this interest in and exploration of Institutional Research and Effectiveness has been an increasing pressure (to some extent self-imposed) to demonstrate quality and effectiveness of distance learning initiatives. Since online teaching and learning is still so new, there seems to be quite a lot of scrutiny. Questions about whether online courses are effective as traditional classroom learning abound.

As a manager of distance learning initiatives, I've struggled to find ways to demonstrate the quality and effectiveness of these programs. I know they're high quality, and I know that the faculty and students involved in them generally find them even more effective than their traditional courses. But how to demonstrate this to the skeptics?

It seems that one direction this has led folks in the field is toward creating frameworks for quality measurement in distance learning. It's interesting to me that rather than doing extensive studies and data gathering about teaching and learning outcomes we've instead decided to create models that help us evaluate the quality of the inputs to distance learning courses. But then, it seems that the history of assessment in higher education is about this balance between evaluating inputs and looking at the outcomes.

More on this in Origins: Part IV.

Origins, Part II: Why "Truth and Beauty"?

This post originally appeared in my short-lived blog Truth and Beauty, and was moved to this blog in June, 2007.

Why call this blog "Truth and Beauty"?

When I was pursuing my Masters at the Harvard Graduate School of Education, I cross-registered for a course in System Dynamics at MIT's Sloan School of Management. I had worked with Peter Senge while at PBS and was intrigued by his idea of systems thinking. I thought that the course would help me better understand this concept and the whole idea of organizational learning.

It did indeed give me insights into Senge's ideas. What I didn't know, though, when registering for the course, was that it was going to be highly technical. We would use specialized software (STELLA and iThink) to build complex business models and run simulations of these models to observe how they behaved over time. There was a lot of equation writing and playing with different data inputs. We were tweaking the models to see what it would take to get them to behave in just the way we wanted them to. It was a lot of fun, and an incredible learning experience.

Toward the end of the semester, our instructor included a lecture on "Truth and Beauty." The models that we'd labored over, he warned, were not to be confused with the truth. They may have been beautiful, telling interesting stories and even giving us some insight into decision-making. There were not, however, reality--something far more complex than could ever be modeled.

I'm fascinated by this paradox about data and decision-making. We often treat data as gospel. But usually there are more questions than answers, when we really look close. So are the data pointing us to the truth, or just beautiful possibilities? Or are these the same?

I guess this is what Keats meant when he said "'Beauty is truth, truth beauty,' - that is all ye know on earth, and all ye need to know." (Ode on a Grecian Urn).

All ye need to know, indeed. There's way more I gotta know, and I'm going to keep looking.

Origins, Part I: Why a Blog for Effectiveness?

This post originally appeared in my short-lived blog Truth and Beauty, and was moved to this blog in June, 2007.

Recently, a confluence of events in my professional life has led me to explore the worlds of educational assessment and institutional effectiveness. I'll describe more about these events in follow-up "Origins..." posts.

Basically, though, I'm hoping this blog can become a resource for folks working in postsecondary education—with responsibility for or just an interest in institutional research/effectiveness, policy analysis, planning, and evaluation/assessment. There are a wide variety of resources out there on these topics, but I haven't yet come across a commentary-type editorial resource. This is not to say I'm the first to be blogging or providing commentary on activities in this world—only that I haven't yet discovered these kinds of resources.

As my disclaimer under "About This Blog" says, I am not by any means an expert in the field. I'm just an academic technology administrator with a great deal of interest in institutional research. Seems like these days, it behooves us all in the field of education to have such an interest... I hope that my naive perspective on all this can be a helpful resource.

I've simply been doing a lot of thinking about the related issues lately (for reasons upon which I'll soon elaborate), and I thought it might be best to do some of this thinking out loud and share it with others.

Welcome to "Truth and Beauty." I hope you find it interesting and engaging.

Re-Invention and Diffusion of Innovation

This post originally appeared in my blog Digital Amalgam, and was moved to this blog in July, 2007.

A while back, a colleague of mine and I wrote about Diffusion of Innovation theory and how it might relate to faculty development. Truth be told, we were familiar with Everett Rogers's work on diffusion of innovations and read a lot of secondary sources related to it, but neither of us had ever picked up the book Diffusion of Innovations and read it. I finally bought it recently, and I'm finding it infinitely readable for an academic book. And the most recent edition just came out, so it's got lots of up-to-date examples and includes references to recent adaptations of the theory, including Malcolm Gladwell's stuff (Tipping Point) and others.

One of the pieces I missed by not having read the book before is the idea of "re-invention." This is how an innovation is changed or modified by an adopter in the process of taking on the innvation and implementing it. Rogers says in the book that many researchers have largely ignored re-invention (so I don't feel so bad for missing it), and that much diffusion research has focused only on innovations that are adopted as developed.

What's interesting to me, working with teachers who are famous for "adopting and adapting" is that it's virtually impossible to track diffusion of innovations among faculty if you don't pay attention to re-invention. Teachers develop their own way of using an innovation so that it fits with their own approach and students' learning styles and abilities.

But it's not just that we should pay attention to re-invention. I think we should encourage it when pushing to have an innovation adopted. I think for the latent adoptors (which Rogers calls "early and late majority"), knowing that re-invention is an option may make the difference between adopting and not.

Let me take online courses as an example. I think most people who haven't taken or taught an online course have a mental picture of what an online course is, and that picture is either wrong or it's very fuzzy. Because early adopters have a high tolerance for ambiguity, they jump in and do an online course even though they aren't clear. For those other folks (I'm talking about teachers of online courses now, not students), we try to develop specific models and formats and templates for an online course--this helps give them a better picture and helps make adoption easier. But once adopted, a good teacher isn't going to want to stick with that format or template. The level to which re-invention is encouraged here could make the difference between that teacher sticking with it or not.

The more I write, the more I feel like I'm stating the obvious here. But this is a new insight that's emerging for me.

Earlier today, I was reading about "folksonomies" versus controlled vocabulary, and I think there's a similar re-invention dynamic here. But I'll have to write more about that later. Gotta get to work!

Back at 'em...

This post originally appeared in my blog Digital Amalgam, and was moved to this blog in July, 2007.

Happy New Year!

Man, did I ever get consumed at the end of the year! Between the new job and the holidays... Yikes! Has it really been six weeks since I last posted an entry? Well, here's my first resolution for the year--get back to blogging!

More to come...

Ending the Tyranny of Online Discussion

This post originally appeared in my blog Digital Amalgam, and was moved to this blog in July, 2007.

Something that's occurred to me during online teaching stints is that not everyone prefers to engage in developing knowledge by talking or discussing issues. This is why there are so many who never raise their hand in class. Are these people not learning? Well, maybe some of them aren't, but I suspect that many of them are. They're just processing information and building their knowledge in a different way.

I know there's all that stuff about the social construction of knowledge--Lev Vygotsky and all that. But does one really need to be a talker in order to learn? Is it possible that some talkers, and some discussion, actually create barriers to learning? I think so. And one of my great frustrations about teaching online is that, while I know online discussions are vital to building community and creating connections between and among students and facilitator, some people just won't get anything out of having to post to a discussion--these are people who feel oppressed (no, I don't think that's too strong a word) by online discussion and who we risk losing if we insist on forcing people to "talk" so much in online courses.

Unless... Maybe we can create discussion guidelines and rubrics that are more sensitive to different ways of processing information. I don't think I'm talking about Multiple Intelligences or Learning Styles here. I'm talking about different ways that people prefer to work with or process information and ideas. This processing is but one path to learning.

So what do I propose? A few years back, I experimented with an alternate model for discussions. Instead of having one rubric for discussion participation, I had four. I tried to recognize that people would have different approaches, and different strengths, in the discussion boards. This experimental model is by no means fully developed, but I described it to some faculty I'm teaching about online facilitation in a discussion board post the other night. I thought I'd share it here.

Please let me know, via the comments link below, what you think. If you decide you're going to give this, or some modified form of this, a try in your own teaching please drop me a note about how it goes. Let's find a way to end the tyranny of online discussion!

Subject: An Experimental Participation Model
Message no. 204 [Reply of: no. 201]
Author: Jim Woodell
Date: Monday, November 15, 2004 8:59pm

As an online instructor, I have also struggled with the
notion that not all students want to participate in the 
same way. However, I've also found that students need 
to be present in the online course and I've found the 
discussions an invaluable assessment tool (so much 
so that I've lightened up on other assignments and 
made participation a huge part of the grade).

But, I developed a model once that I hoped would 
addressthe issue that not everyone wants to 
participate by "talking." Unfortunately, much of my 
documentation of this model disappeared when I 
lost a hard drive, but it went something like this:

At the core of the model was the idea that there 
would be four different types of participation, and 
students should select one of the types based on how
they felt they would be most likely to engage with the 
content. Each type of participation had a rubric to 
provide students with characteristics of that type, and 
also an example or two.

The first type of participation was for the "talkers"--
people who liked to engage in conversation and who 
learned a lot through this kind of interaction. This type 
of participation looked very much like what we've done 
in this course--exchanges of messages, back-and-
forth and tying ideas together.

The second type of participation was for "thinkers"--
people who were more reflective and who liked to look 
at a lot of different ideas and consider their own 
perspective on those ideas. This kind of participation 
required students to go out and seek new ideas to 
introduce to the discussion. The ideas could be from 
web sites, articles, etc. They could even choose to 
highlight an idea that had been presented in an assigned 
reading or in the lecture notes, but that wasn't seeing the 
light of day in the discussion. Thinkers were not required 
discuss the ideas--only to post the ideas and to say how 
it changed their thinking. 

The third type of participation was for "workers"--people 
who were most engaged when they were trying to apply 
an idea or concept to their own experience. Again, these 
folks didn't have to get involved in the discussion, but 
they did have to post something to the discussion board. 
What they were asked to post was and example of how 
one of the concepts we were talking about had played 
out in their own experience--how it applied to them.

Finally, the fourth type of participation was for "lookers"
--people who liked to watch the trends and patterns of 
what others were talking about (or thinking, or working...) 
and to point out the trends and patterns. These folks were 
required to post analyses of the discussion or the course 
content, using tables or charts or simple outlines. 

That's the idea in brief. Unfortunately, I've not had a 
chance to test this but for a brief period of time in one 
of my courses. One thing I remember discovering was 
that, surprisingly to me, everyone ended up trying out all 
the different kinds of participation. Though some 
claimed to be solidly in one camp, they were often 
inspired to try the other participation modes.

Beyond Content

This post originally appeared in my blog Digital Amalgam, and was moved to this blog in July, 2007.

...and beyond teaching?

Yesterday, I said that I thought that too much emphasis does seem to be placed on getting the content out there and not enough, perhaps, on the teaching or facilitation around that content. While I think there's no replacement for a good teacher to help someone progress through their learning, Derek Morrison at the Auricle blog introduced me to a different way of thinking about the "content problem" that Sebastian Fiedler wonders about.

Auricle: "But we all know (don't we?) that a load of content online, no matter how prestigious the source, does not e-learning make; a fact recognized by David Wiley's Open Learning Support (OLS) project, a pilot research project launched last April in collaboration with MIT."

As I said, it's not the content that makes MIT. Then I said, "it's the teaching." Well, that was probably a simplification. It's the teaching and learning. And while we can't re-create the MIT experience around it's content, the Open Learning Support project fascinates me as a way to at least create some additional, and I think needed, piece of the puzzle--a learning community. What a great idea to create an open space where those folks who want to take advantage of the MIT content can also engage with other learners!

We educational progressivists like to tout the notions of learner-centered classrooms and constructivism, and maybe this project, with its "self-organizing" learning communities will end up showing just how powerful such ideas can be. But will these open communities really take us not just beyond content but also beyond the need for teaching?

I don't think so. Derek also references Gilly Salmon's 5-stage model, suggesting (I think) that this kind of progression may be missing in some of the OLS project communities. In order for a learning community to advance, it needs someone to tend it. It may be that effective self-organizing learning communities can be successful if someone (or a few someones) in the community take on a facilitative role.

But it does feel to me like content plus community are still not quite enough. Facilitation seems to me to be key. I'm eager to see what the OLS tells us about this!