Maybe you’d design a model-focused course that emphasizes laboratory experiences? And to that you might add video analysis; it’s just cool for students to take and process their own data. In my opinion, video analysis also helps students experience physics in a more authentic way. You might also have them write a few simulations that model the physical phenomena they are learning about. This gives students another “cognitive hook” to use when thinking and talking about physics.

And how many students are you teaching in this theoretical class? 10? 20? maybe 40? What about 40,000? Or more?

A MOOC with a lab

As I mentioned in an earlier post, I am helping to do research in a massively open online course (MOOC). Our course is called “Your World is Your Lab”. By leveraging modeling, video analysis, and numerical computation, we are attempting to provide this type of authentic, scientific experience. Here’s the intro video for the course:

Now I am very skeptical of this MOOC business. But, I think it’s also important to know a lot about something before dismissing it entirely. That’s why I’m deeply involved in the planning of and research into this MOOC.

“Your World is Your Lab” is a different type of MOOC because it contains a home laboratory component. Students go out into the world, take video of some phenomenon, analyze it, and construct a video report of their observations and analysis. These reports are graded by their peers (i.e., other students taking the MOOC) and these laboratories constitute the major portion of the “grade” in this MOOC.

My main issue with MOOCs is that they perpetuate the transmissionist model of education (see below).

I think laboratory experience helps to bring in the constructivist model and by coupling that with “peer review”, we actually emphasize that doing physics is a social act.

Unfortunately, that is where it ends. Lectures are pre-recorded, interactions with the teaching staff are limited, and homework and exams are computer-based. So, we will have a pretty cool lab experience (if people can figure it out) and an average lecture experience.

Participation will vary greatly, and it’s unclear how many students (and how often they) will perform the laboratories. This is an ambitious undertaking; porting an entire introductory physics course to the MOOC format. That is why the research is so important.

Major media outlets (e.g., David Brooks in the NYT) are touting the coming digital education revolution, but it’s unclear if students will actually participate when the course looks like a real college course. Equally unknown is which students complete the course, what factors influence their success, and what they learn from these courses. To start answering these questions, we are collecting a variety of demographic data, answers to conceptual assessments, and affectual measures from students taking our MOOC.

I hope that some of you will sign up for the course and tell me what you think. Oh, and if you are a high school teacher who completes the course, you can earn 6 continuing education credits from AAPT.

In it, Mr. Jacobs describes his experiences with eleven different MOOCs. The beginning of his piece bothered me quite a bit because it read like the same old stuff. Early on he says, “I’m getting Ivy League (or Ivy League equivalent) wisdom free. Anyone can, whether you live in South Dakota or Senegal, whether it’s noon or 5 a.m., whether you’re broke or a billionaire.” It’s unclear that impoverished Senegalese are completing MOOCs en masse. More likely, white men from the US, Canada, and Europe are the certificate earners.

But later in the article, he gives a thoughtful critique of each aspect of his MOOC experiences. The most telling part is the low grade he gives to instructor-student interactions. Most instructors can hardly manage a class of 200, let alone a class of 40,000. MOOCs have very little instructor-student interaction. It’s a one way conversation.

Jacobs believes that, “[f]or MOOCs to fulfill their potential, Coursera and its competitors will have to figure out how to make teachers and teaching assistants more reachable. More like local pastors, less like deities on high.” That is, to do anything meaningful, MOOC providers must abandon the transmissionist model of education and develop technologies to facilitate that change. He goes on to mention that providers are looking to enhance instructor-student interaction.

But, what will the increased cost of those interactions be? Could MOOC providers produce something close to the brick-and-mortar experience? I’m still pretty skeptical. But then again, I love my ivory tower.

• Steve Spangler (who lives in the Denver area!) dropped by for the keynote address.He showed us a few demonstrations and told us how he got into TV. It’s a really cool story involving someone from NBC asking if he wanted to be the next Don Herbert (Mr. Wizard). Apparently, he called Don and asked him for advice when he first started. Don said, “Don’t let the bastards put you in a lab coat!”, which means try to make science accessible. Don’t look like a “scientist.”He also told us about the time he was questioned by police for blowing up bottles of liquid nitrogen. Here’s the video of that:
  
  I had not heard of Steve Spangler before this meeting, but I will definitely be using his YouTube channel for my future demo ideas.
• My buddy, Ariel, gave a great talk about PhET sims. He talked about how sims are designed and why they are designed they way they are. He left folks with a few best practices. He said, “just let students play with the sims for 5 minutes.” You won’t have to tell them where all the controls are. They’ll figure it out themselves.
• I gave on talk about the Global Physics Department that was well received. I have a feeling there’s going to be a lot of new blood joining the meetings. Here’s a copy of my powerpoint slides for anyone interested.
• Stephan Graham (Arrupe Jesuit High School) gave a fantastic talk about how he teaches the language of physics using literature. He teaches at a predominately Latino school where more than 50% of students are English Language Learners. Stephan uses short stories in his physics classes to get students to identify when “physics” language is used in a physics context and when it is used in a figurative/literary context. Very cool stuff. Here’s the list of short stories he recommended: Bill Naughton’s Spit Nolan (velocity and acceleration), Liam O’Flaherty’s The Sniper (projectile motion), Francisco Jimenez’s The Circuit (series and parallel circuits), Michael Cunningham’s White Angel (impulse and momentum), and Kurt Vonnegut’s Harrison Bergeron (force).
• Finally, the whole meeting ended with the classic sledgehammer/nail bed demo. Here’s a video I shot.
  

I’m really sad to be leaving this section. I’ve been to only two meetings, but I can tell it’s a really tight-knit group that is doing a lot of great things in the physics education community.

The problem is that the MOOC phenomenon is the higher-education equivalent of “Keeping up with the Joneses.” Universities are getting involved because others are, but it’s unclear what students are learning (in anything) in these new environments.

Now, I don’t want to come off as critiquing something that I know very little about. In a future post, I will talk about my involvement in the development of and research around an introductory physics MOOC. But for now, let’s just talk about what folks are saying about MOOCs, and what the reality seems to be (so far).

Can we transmit knowledge?

There are a number of articles written by well-known journalists on the “MOOC revolution.” But, most of them seem to say the same thing, universities are too darn expensive and MOOCs can help change that. Now, those statements are connected by a tenuous strand of reasoning: MOOCs can reproduce a university education. David Brooks goes into detail about what online education might be able to do for students.

But, I think he misses the point entirely. Education is not just about transmitting knowledge for preparing the workforce. It’s also about empowering individuals to live in the world and empowering society to meet the challenges of our modern age. Education is the means by which society replicates itself. It is how we move forward together. A college education does not just prepare you for a job, but it prepares you to interact with the world, to process complex information and make decisions based on that information, and to engage in societal discussions.

Sure, you might call some of this “practical knowledge” (as Brooks seems to), but his framing that such knowledge is imparted or absorbed says a lot about what he thinks education is. To him and others writing about MOOCs, education is the transmission (or absorption) of knowledge. That is why it is so easy for him and others to accept the MOOC model. If we can simply figure out how to transmit this knowledge into the student’s mind as efficiently as possible, we can “educate” millions.

It’s premature to think that MOOCs can or will replace brick-and-mortar institutions. In 1913, Thomas Edison said of the motion picture, “Books will soon be obsolete in the public schools. Scholars will be instructed through the eye.” It’s been a century and that still hasn’t happened. Why? Because there’s much more to education than just transmitting knowledge.

But what does the data say?

The data from MOOCs are just coming in. Over the next couple of years, we will collect more data that will help us explain what these new environments are doing for students.

The best preliminary data that I have seen so far comes from this presentation by four computer science professors. It shows the following:

• There appears to be a well-established power law drop off for MOOC students. Only about 5-10% will earn a certificate, that is, complete the course successfully (Slide 4).
• MOOCs are not serving a diverse audience. Certificate earners are predominately middle-aged, white males (Slide 31).
• MOOCs are serving those who already have degrees (perhaps degrees in the field). The vast majority of certificate earners tend to hold at least a bachelor’s degree (Slide 32).

Now, these conclusions are based on a handful of courses. So that doesn’t mean such results will hold. Moreover, it’s not clear from these data what certificate earners have learned from these courses. There’s no independent measure of learning in these courses, we just see what students completed as part of the course.

My involvement in an introductory physics MOOC attempts to answer some of these questions, at least for a physics course marketed to first-time physics students. We are collecting much more demographic data from students and giving pre-post assessments that can be compared to brick-and-mortar performance. I guess we will see what happens in the fall.

Feel free to leave your comments on MOOCs below.

I have stayed active on Twitter. I’ve met some new people and had some great conversations along the way.

Blogging hasn’t been a priority because I’ve been trying to write a bunch papers, which I have.

And land a permanent job, which I have.

So next fall I’ll be assistant professor of physics at Michigan State University.

— Danny Caballero (@physicistdanny) April 12, 2013

And take care of a new baby, which is going OK.

I have two kids now. I finally understand why my parents drink.

— Danny Caballero (@physicistdanny) March 5, 2013

But I’m going to try to write a bit more. There’s a lot of things that I’ve saved up and would like to share. So, let’s see what happens.

Writing functions,

Using FindRoot,

and Writing for loops.

Ben Zwickl has posted more of his lab driven screencasts on our YouTube Channel, also. Feel free to use these in you classes. And I promise, more interesting posts are coming.

Matt Kohlmyer, Michael Schatz and I wrote a paper comparing introductory E&M curricula using the Brief E&M Assessment (BEMA) in the early part of my career. You can find it here. If you want to review the BEMA, there’s a nice article here. Contact me directly if you’d like a copy of the BEMA.

In that article, we found that students taking Matter and Interactions (M&I) E&M performed better overall and on individual topics than students taking a pedagogically reformed, but traditionally sequenced E&M course at several different institutions. This led us to conclude that students taking Matter and Interactions were better prepared in E&M (as measured by the BEMA). The BEMA was specifically designed as the lowest common denominator concept inventory for E&M; hence, our conclusions were fair and justified.

In a new article that we have submitted to AJP, we performed a very similar comparison between mechanics courses: Matter and Interactions mechanics and the equivalent traditionally sequenced course. In this work, we used the Force Concept Inventory (FCI). Why? Because we had anecdotal evidence that Matter and Interactions students underperform on the FCI. This article is likely to be published in AJP which is not an open-access journal. I’m posting the preprint of the article in this post. But, it will eventually be on the arXiv.

Preprint of FCI paper

We found that students in M&I mechanics do underperform compared to students taking a traditionally sequenced course. However, in this article we did not conclude that traditional students are better prepared in mechanics. Why?

The FCI was not designed to compare curricula (or pedagogy, although it’s often used for this purpose). Moreover, the FCI is not aligned with the content and goals of the M&I course. One of the first things I learned about educational reform is the alignment of instruction and assessment. Clearly, the instruction in this case is not aligned with the FCI. In fact, the creators of M&I have clearly stated that this curriculum has fundamentally different goals (e.g., a 20th century introduction to mechanics) from a traditional course (e.g., 17th-18th century). We attempted to communicate these ideas in an earlier draft of the article.

When the preprint (and my thesis work) was originally picked up by the blog-o-sphere, our message was lost. Mark Guzdial, a member of of my thesis committee and eminent CS-ed blogger, concluded that “Computation + X doesn’t necessarily mean better learning in X“. While this might be true, it’s a terribly hard thing to measure. Nor do I believe my work in this area can make that conclusion. The whole course, not just the Python programming, is under consideration. Chad Orzel picked up an interesting thread, “practice matters.” That is quite true, but what is also important is the choice of evaluation instrument. To be fair, Chad makes a number of interesting remarks concerning the purpose of the introductory course. Is it to improve FCI scores? I think not.

In the above version of the paper, we have carefully laid out our message: concept inventories (in general) are limited. Concept inventories that are not designed with the content and goals of the course in question in mind are useful tools, but do not provide a complete picture of the course or what the students have learned. It’s very important that the evaluation match the instruction, otherwise, what are you measuring? Moreover, concept inventories are only able to compare students on concepts, methods, and tools which curricula both treat with equal intensity (like our BEMA work). Otherwise, there are caveats on the results.

Concept inventories also miss what are likely the most interesting aspect of curricular reform, the effect on new content, goals, and methods. How do we value these new effects? How are they weighed? Can we measure this? Such questions are important for those considering new ideas in their courses. And these are not easy questions are to answer. Nor are the answers fixed, but dynamically changing as we start to understand the purpose of our introductory courses.

In light of the underperformance by M&I students, is anything changing? Sure. My Georgia Tech collaborators are trying work within the curriculum to shore up conceptual difficulties. But, major changes are not planned. This is because we value the new concepts, methods, and tools of the M&I curriculum over a single measure of student performance.

I did write this little Python script which uses a very basic relaxation method to compute the temperature field on a 2D surface. I plan to convert it to MMA over the break. Enjoy!

[Download]

from __future__ import division
import matplotlib
matplotlib.use('MacOSX') ## Change if you are using a different backend
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm
import matplotlib.pyplot as plt
import numpy as np

def SolveLaplace(nx, ny, dx, epsilon = 1e-5, imax = 1000):

    ## Initialize the mesh with some values
    T = np.zeros((nx+1, ny+1))

    ## Set boundary conditions for the problem
    T[0,:] = 100 ## Top Boundary
    T[nx,:] = 100 ## Bottom Boundary
    T[:,0] = 0   ## Right Boundary
    T[:,ny] = 0 ## Left Boundary

    ## Store previous grid values to check against error tolerance
    TN = T + np.zeros((nx+1, ny+1))
    err = TN - T

    ## Constants
    k = 1          ## Iteration counter

    ## Iterative procedure
    while k <= imax:

        for i in np.arange(1., nx):

            for j in np.arange(1.,  ny):

                TN[i,j] = (T[i-1,j] + T[i+1,j] + T[i,j-1] + T[i,j+1])/4.
                err[i,j] = np.abs(TN[i,j] - T[i,j])

        T = TN + np.zeros((nx+1, ny+1))
        k += 1
        errmax = np.max(np.max(err))

        if errmax < epsilon:

            print("Convergence after ", k, " iterations.")
            return T

    print("No convergence after ", k, " iterations.")
    return False

def PlotSolution(nx,ny,dx,T):

    ## Set up x and y vectors for meshgrid
    x = np.linspace(0, nx * dx, nx+1)
    y = np.linspace(0, ny * dx, ny+1)

    fig = plt.figure()
    ax = fig.gca(projection='3d')
    X, Y = np.meshgrid(x,y)
    surf = ax.plot_surface(X, Y, T.transpose(), rstride=1, cstride=1, cmap=cm.cool, linewidth=0, antialiased=False)
    plt.xlabel("X")
    plt.ylabel("Y")
    #plt.zlabel("T(X,Y)")

    fig2 = plt.figure()
    cs = plt.contourf(X, Y, T.transpose(), 32, rstride=1, cstride=1, cmap=cm.cool)
    plt.colorbar()
    plt.xlabel("X")
    plt.ylabel("Y")

    plt.show()

## Size of plate and mesh
nx = 32
ny = 32
dx = 1/128

epsilon = 1e-2 ## Absolute Error tolerance
imax = 10000    ## Maximum number of iterations allowed

T = SolveLaplace(nx, ny, dx, epsilon, imax)
PlotSolution(nx, ny, dx, T)

And here’s the visual output for two sides held at 100 degrees.

There are a number of things that we try to do in teaching: introduce new concepts, convey interesting information and provide students with practice using new tools and concepts. However, one practice that can make teaching very challenging (and might be neglected) is developing students’ habits of mind.

The tacit knowledge which we use daily to solve problems and design experiments is rarely communicated directly to students. In fact, as a student, I thought that many of these habitual practices were “tricks”. By the way, we don’t do them any justice by calling them “tricks”.

One such nugget of tacit knowledge is how we as physicists use Taylor series. Actually, we are often using MacLaurin series; I’ll come back to this in a moment.

We hoped to communicate some of these ideas early in the semester. We had developed a few clicker questions and a tutorial.

On the tutorial, most students demonstrated a good working knowledge of Taylor (MacLaurin) series (although they faltered a bit with interpreting their results). This might be because they had just answered several clicker questions and an example problem. Or it is likely because the tutorial had been altered (by the instructor) to include far more calculations, so they just got better at performing calculations. In this tutorial and on their accompanying homework, students received a lot of practice cranking through Taylor (MacLaurin) series. Some problems had a physical context, some didn’t.

However, many students were unable to perform four key functions:

1. Identify the small parameter in the equation to be expanded
2. Execute the expansion efficiently
3. Compute a Taylor (not MacLaurin) series
4. Interpret the results

Now, only the first three of these are related to Taylor series. The last one is part of a larger problem which students at this (and other) level exhibit, the disconnection between math and physics (saved for a future post). I’ll discuss the first three, each with an example.

Identify the small parameter

Consider the following equation which compares the acceleration due to gravity between the surface of the Earth, , and some distance above it, .

Students were given an equation similar to this and asked to determine the value of in that was near the surface of the Earth. A student (we’ll call him “Lionel”) came to my office to ask about this problem.

Lionel works with a large (~10) group of students and mentioned that his study group was unable to satisfactorily answer this question. I asked what he and his group tried to do. He immediately said they had no idea which variable was the “right” one and they tried , and , but the group wasn’t sure what to do. The group was unable to identify the small parameter and thus tried all the possible variables. In addition, they were at a loss as to what value to expand about, they decided that zero was the logical choice.

Each of these variables could be used to determine a Taylor expansion of the function , but which one is the right choice?

Most physicists would form a small parameter given the context of the problem. In this case, by dividing out in the first term, you’ll get a small parameter . You can then compute the MacLaurin series in , because this value is close to zero. It’s also possible to use , if you compute the Taylor series about .

Executing the expansion efficiently

One of the major benefits that I see to being a physicist is the ability to build a quick model and get an approximate answer quickly. So, I’ve committed a few expansions to memory (e^x, sin x, cos x, ln(1+x), etc.). Our students are still learning the benefits of this tacit knowledge.

When our students compute a Taylor series (after they’ve gotten through the mess of identifying the small parameter), they tend to use the full form of the Taylor series.

Now, this method will always work. But it can be very inefficient.

Suppose a student has been asked Taylor expand a function that is predominantly polynomial, maybe with a ln(1+x) added to the end of it. Such an equation appears if you consider the vertical position of a particle that experiences linear drag as a function of its range.

I interviewed a few students who were attempting to Taylor expand this function. All but one (who was a very strong student) tended to take derivatives of the entire function and evaluate them. Most physicists have adopted the technique of searching for the non-polynomial terms and replacing them with the first few terms of the recalled expansion. The binomial expansion was my savior for graduate E&M.

While students might eventually produce the correct answer (after a significant amount of algebra; none did in my interview), such a habit is good for physics students to acquire. It will make their life easier and enculturate them into the spherical cow club.

Perform a Taylor Expansion

Many of the examples of Taylor expansions in undergraduate studies tend to actually MacLaurin series (a Taylor expansion about 0). So when students are asked to compute a Taylor series then tend to fail.

Now, you might think, “well, if all the examples are around x=0, why should we bother with others.” Changing the point about which we compute the Taylor series exposes the fragility of students knowledge of Taylor series. In addition, many problems in science and engineering do not produce nice functions where the expansion variable or the point about which the expansion should be performed are immediately clear as our cooked-up examples. So, a thorough working knowledge of Taylor (not just MacLaurin) series is important.

On a recent exam, we asked students to compute the Taylor expansion (up to the first non-vanishing, non-constant term) of a position function about some non-zero value for the time, .

Roughly 50% of the students computed the expansion about time equals zero; many of whom plugged in the non-zero time value to obtain a function with no time dependence. Only 10% of students solved this problem correctly.

What does this all mean?

For me, this is a lesson in the failure to fully explicate the tacit knowledge which physicists use that we want our students to employ.

These three problems might be mitigated by additional activities: Students might explore MacLaurin expansions by choosing different parameters and evaluating they expansions utility. They might evaluate the efficiency of computing a Taylor series using the formal method and the “search and expand” method. Finally, some experience with Taylor series around non-zero points would not be the worst thing in the world.

I’m planning to investigate each of these issues in detail with a few more interviews and some new questions. These investigations will hopefully codify students’ challenges with Taylor series and help to improve the quality and focus of activities that we are developing to meet these challenges.

But, ultimately this work begs the larger question, “what tacit knowledge do you assume your students are using?”

In talking with Phil, I notice that Google employs different model from a traditional “research-based” approach at work for the development of curriculum. Not better or worse, just different. This leads me to ask the question: Should we crowd-source curriculum development? And what does that mean for those of us who work in educational research?

A new model for curriculum development

In my view, Google’s philosophy for computational modeling curriculum development employs four ideas: open, transparent, community-driven, and peer-reviewed. Their plan is to provide a venue for curriculum materials to be posted, commented on, modified and reposted. Phil is developing some of the initial materials to jump-start the efforts of this community of users (I plan to contribute when and where possible). In this way, Google follows the commitment to transparent development which has made the open-source community so successful.

Take GNU/Linux, Apache, and Python as examples. Linux has grown from the domain of geeky sys-admins to roughly 2.5% of the market share. Apache has grown to serve up more than 60% of world’s websites. Python is quickly becoming a language that is used across domains (e.g., web apps, desktop apps, and science). This development model has proven to be successful for Google. Many people use the Google App engine to develop their web apps or code.google.com to host their software projects (including me).

Can this same model be applied to curriculum development?

Sure. In fact, many of the features of this model are present in the more traditional “research-based” approach. Or at least, how I’ve tried to perform educational research.

I’ve (and others) attempt to make our work as open and transparent as we can. This can be blogging about the work, posting materials that have been developed, or contributing to community through research talks and posters. Each of these contributes to an open and transparent development environment. Moreover, all software and materials that I’ve worked on are licensed through the GNU GPL v3 and are free to use, distribute, and develop.

The community (i.e., the physics education research community) does drive investigations through initiatives and funding opportunities. But these initiatives are more formal and less organic than the approach which Google employs. The community also offers peer-review of ideas, materials, and results. If you’ve ever attended an AAPT national conference or written an educational research paper, you can certainly attest to that.

What is novel about the Google approach is that it crowd-sources initiatives and peer-review to the community of users. This model is an exciting approach for those interested in quick feedback and spirited discussion.

It might be difficult for some (not me), especially those whose positions are dependent on funding and research publications (that’s me), to accept this model of curriculum development.

What’s missing and what we might do about it

This model does not appear to include traditional evaluation using research studies or the development of measurement instruments. It’s strongly organic and driven by educators (typically, not educational researchers). Neither of these is a bad thing; educators know what’s working in their classrooms and what’s not. I know that feeling having taught both a small AP class and a large college class. Believe me, when it’s not working, it feels worse in a small class.

The mechanism for development and dissemination which Google plans to employ reminds me of Field of Dreams. Build it and they will come. That’s true, but not without some advertising. I propose the hosting materials on Google servers rather than linking to materials in their forums. A central repository will make it easy for folks in the community to find and share materials. Think comPadre instead of your personal website.

There should also be a method for commenting and rating materials which others develop. For an example, see this activity for the Moving Man PhET. Detailed discussions and comments of blogs posts can be linked to these materials. This mechanism will give credibility to the materials and provide additional peer-reviewed feedback for the development of new materials.

The features that do not overlap (e.g., research studies and measurement) might have to do more with what funding agencies want and what journal publications require than what benefits students. Google is not in the business of educational research, they are trying to contribute to the conversation and to help steer efforts towards novel ideas. And ultimately, it is teachers who will decide whether to use these materials and how to modify them. Choice is very important. But, there might be a way to resolve these two seemingly divergent models.

Merging crowd-sourced and research-based

Notwithstanding the two ideas I proposed, we can further bridge the gap between crowd-sourced and research-based development. It’s clear that both can be open and transparent. For those of us who must publish or perish, it’s possible to develop open content, distribute it and test it systematically (as we have and continue to do). Such evaluations (or publications) could be linked to forums or material posts for users who are interested in the technical details. We can summarize our findings in blog posts (as I have done).

Using these mechanism, the most useful (as judged by users) and well-researched (as judged by specialists) material can be showcased.

For educational funding agencies, this is new and divergent way of thinking about research. Peer-reviewed publications are not the sole mechanism by which research is developed; social networks, blogs, and other networked forums are fast becoming the way in which we communicate and share ideas. They’d be smart to encourage such open collaboration and communication.

I’m sure most people have seen this comic.

While the situation is not as dire as that, it can be pretty close. The crowd-sourced model provides a mechanism to get results and feedback before attempting to get funding. You haven’t done the science yet, but you’ve vetted your ideas with users.

Is this really curriculum development?

Some of you might have noticed that I’ve been talking about curriculum materials (i.e., lessons and activities). Lessons and activities do not a curriculum make.

So, in the formal sense, Google is not engaging in curriculum development. They are providing resources and developing activities (and encouraging others to do so).

This is where education researchers might find a place; developing theoretical and practical frameworks around which we (as a community) might build a computational thinking curriculum.