Edcamp Study Process Reflections

Do Edcamps do what we think they do?

This page is an exercise in self-regulation and process documentation. It will show the source and evolution of ideas and articulate some of the strategies used to manage the project. Project reflections appear in regular text. Self-regulation reflections appear as block quotes. Entries appear in chronological order.

January 31, 2016

This entry is intended to give some background to the origin of the idea for a study into EdCamps. I have attended several Edcamps and organized a couple as well. It is a very attractive form of professional engagement but what is it really, and what is it intended to do?

Traditionally weekend events, Edcamps draw people who are interested enough in teaching and learning to give up the better part of a Saturday to talk with other teachers. While there may be a theme for the Edcamp (administration, technology, art, middle years, etc.) the actual content is not determined until the morning of the event and is driven solely by the participants. This raises a couple of interesting questions: Is this professional development? Is it more appropriately called professional learning? Or maybe professional engagement?  Does it really matter what it's called or what it accomplishes?

O'Brien and Jones (2014) discuss the debates and decision making that went into naming an academic a journal that explored professional development/learning. In it they suggest that "Professional Development" (PD) is closely related to institutional requirements, directives, and initiatives. "Professional Learning" (PL) on the other hand, is more individual. Pentland (2014) differentiates "Professional Engagement" from "Professional Exploration". Engagement occurs among close colleagues and familiar topics. "Exploration", Pentland says, occurs with individuals outside our usual realm of experience; it pulls ideas in from beyond.

Whatever we call it, Edcamps have received a warm embrace by the education community. At the time of this writing (Jaunary 31, 2016) there are well over 1000 Edcamp events from countries around the world listed on the Edcamp wiki since the first in May of 2010. As a free weekend event, it draws a self-selected group of keen and engaged participants who extend invitations to share experiences and explore possibilities in their chosen field. With this in mind, it seems the stakes are low: if participants learn something that's good, if not, it's no big deal. There was no investment made, no directive from an employer, no initiatives that required attention. Participants have their own goals and it's up to each individual whether they are met or not.

Some districts and school divisions are also embracing the model offering Edcamp experiences during the school day. These events now have significant cost attached the greatest of which are the wages for all participants. If we are concerned about accountability and value for the public dollar, which we should be, there is a need to determine the benefits of these experiences. Do Edcamps have any measurable benefit to education?

 

February 2, 2016

Gathering Data

It is crunch time; Edcamp MY7Oaks is in three days and I'm focusing on the minutiae of event planning. What equipment needs to be where and who will set it up. What needs to be said when and who will say it. There are many jobs to do and I used my lists to set goals for today. I had some time at lunch and a double prep period today at school but there are always interruptions so I chose some low cognitive-load tasks: cutting laminating, printing room signs, sorting name tags by school and last name. These are the kind of tasks from which I can be pulled and return easily - important when interruption is likely.Tonight I'm working on the survey questions which require a great deal of reflection, analysis, care of wording, and both comprehensiveness and berevity. This task was best done at home where I could close the office door and focus for a few hours.

I have explored a couple of alternatives to get participant responses - the graffiti wall is a great idea and will likely elicit key terms and phrases related to participant experience though I suspect it is less likely to expose negative experiences because of its' public nature. I do think it could serve as a motivator and positively influence others' perceptions when they are exposed to positive messages related to the event.

There also should be a significant collection of social media posts with our hashtag as well as notes on digital shared spaces available for content analysis. I suspect this data will be more content focused as people share key summaries of their learning and experiences. This data may be used to identify popular (meaningful?) experiences and ideas that emerge throughout the day. The notes in particular will reveal who was at which session (if they signed in) and what was discussed. This could point to the introduction and evolution of ideas over a session and, perhaps, throughout the day if similar themes appear in different session notes.

Another quick-and-dirty data collection strategy I encountered at a conference recently used a graph and sticky dots. Participants considered where they fit best on the x- and y-axis variables and stuck a dot at the intersection. You end up with a scatter plot showing the degree of correlation between the two variables. I'll have the graph by the door so people can pop their sticker on the graph as they are leaving at the end of the day. I'll be using this to look at social connections made, and ideas encountered by participants.

As for an end-of-event survey, it affords a degree of anonymity for respondents that is more likely to reflect their actual experience. I've been scouring the literature for previous studies that measured social engagement, information exchange, and professional growth and there have been a few that offered some ideas about what to measure. One framework comes from Vuorikari and Scimeca (2013) research on social learning in a European network of teachers. They offer a sort of hierarchy of social engagement that is a useful model for understanding the intensity of relationships that may emerge from an Edcamp

No one likes to be held back at the end of the day to do evaluations so I really need to be brief. There are four categories I need to assess:

  1. exchange of ideas,
  2. professional networking,
  3. professional learning, and
  4. general questions about affect and intention.

 

Below are the questions I've crafted as well as the intention behind the question. I limited the survey to 12 questions, three for each category. A short survey is more likely to get responses though it lacks depth. I intend on using a 7 point Likert scale asking participants to indicate the extent to which each phrase is true of them:

  • Very untrue of me
  • Untrue of me
  • Somewhat untrue of me
  • Neutral
  • Somewhat true of me
  • True of me
  • Very true of me

I'm choosing Likert scale responses because it returns ordinal data which will can be compiled and reported in a variety of ways. Fowler (1995) describes the value of such rating scales in determining general sentiment for or against the prompt however, he does caution against placing too much confidence in the extent of the differences. As a broad measure, though, I believe this will suit my purposes.

While searching for Likers scale anchors, I found a 2005 article by Elizabeth Fanning on formatting surveys. She offers research-based directions on the physical layout of a survey. Prior to reading the article, I mocked up a survey sheet in what I thought was a very efficient layout. I used a table with the questions on the left and a grid with the response categories laid out at the top. I learned that respondents find grid layouts overwhelming, and that responses should be listed vertically. Questions should be in a darker font than the responses, and that they should be grouped thematically with titles. The page should be one-sided, portrait and unfolded. Shading behind the questions helps respondents focus. Here is the revision that I'll test with some colleagues tomorrow.

Initial design for survey before reading Fanning, E. (2005). Formatting a paper-based survey questionnaire: Best practices. Practical Assessment, Research & Evaluation, 10(12), 1–14. Retrieved from http://parkdatabase.org/files/documents/2005_Formatting-a-paper-based-Survey-Questionnaire_E-Fanning.pdf

Initial design for survey before reading Fanning, E. (2005). Formatting a paper-based survey questionnaire: Best practices. Practical Assessment, Research & Evaluation, 10(12), 1–14. Retrieved from http://parkdatabase.org/files/documents/2005_Formatting-a-paper-based-Survey-Questionnaire_E-Fanning.pdf

Layout changes to reflect best practice from Fanning (2005) article on formatting surveys.

Layout changes to reflect best practice from Fanning (2005) article on formatting surveys.

Survey Questions and Rationale

Exchange of Ideas  
1.     I had casual conversations with people from schools other than my own Targeting EXPLORATION

  • To what extent are participants getting to know others, having casual conversations, learning the "other story".
  • Conversations outside the context of focused sessions.
  • Social conversations where people learn about each other and their situations.
2.     During sessions, I had conversations on topics related to teaching and learning. Targeting INVESTMENT

  • -  To what extent are participants engaged in conversations about teaching and learning, taking in and processing professional content.
  • -  Conversations within sessions that are content focused.
  • -  Professional conversations where people learn about and explore specific ideas related to teaching and learning.
3.     In casual conversations, I shared with other people the ideas I encountered today. Targeting IDEA SPREAD

  • -  To what extent are participants engaged in talking about newly encountered ideas.
  • -  Conversations outside the context of focused sessions.
  • -  Social conversations where people share what they have learned or experienced at the event.
Professional Networking  
4.     I exchanged contact information with people today for the purposes of future professional interactions Targeting BUILDING LEARNING COMMUNITY

 

building connections (adding new people to the network so that there are resources available when a learning need arises)
Rajagopal, K., Brinke, D. J., Van Bruggen, J., & Sloep, P. B. (2012). Understanding personal learning networks: Their structure, content and the networking skills needed to optimally use them. First Monday, 17(1), 1–12. http://doi.org/10.5210/fm.v17i1.3559

5.     I connected with people today that I haven’t seen for some time Targeting MAINTAINING LEARNING COMMUNITY

 

maintaining connections (keeping in touch with relevant persons) (Nardi, et al., 2000; Nardi, et al., 2002)

6.     I sought out a specific person today for the purposes of professional interaction Targeting ACTIVATING LEARNING COMMUNITY

activating connections with selected persons for the purpose of learning (Nardi, et al., 2000; Nardi, et al., 2002)

Professional Learning  
7.     I learned things today that will improve my professional practice. Targeting IMPROVED PRACTICE

Intensive professional development, especially when it includes applications of knowledge to teachers’ planning and instruction, has a greater chance of influencing teaching practices and, in turn, leading to gains in student learning.

Darling-Hammond, L., Wei, R. C., Andree, A., Richardson, N., & Orphanos, S. (2009). Professional learning in the learning profession : A status report on teacher development in the United States and abroad.

 

8.     I learned things today that will improve student learning. Targeting STUDENT LEARNING

Research suggests that professional development is most effective when it addresses the concrete, everyday challenges involved in teaching and learning specific academic subject matter, rather than focusing on abstract educational principles or teaching methods taken out of context. For example, researchers have found that teachers are more likely to try classroom practices that have been modeled for them in professional development settings. Likewise, teachers themselves judge professional development to be most valuable when it provides opportunities to do “hands-on” work that builds their knowledge of academic content and how to teach it to their students, and when it takes into account the local context (including the specifics of local school resources, curriculum guidelines, accountability systems, and so on)

9.     I learned things today that will contribute to my school’s priorities and goals Targeting SCHOOL IMPROVEMENT

Research suggests that professional development tends to be more effective when it is an integral part of a larger school reform effort, rather than when activities are isolated, having little to do with other initiatives or changes underway at the school.

General          
10.                        I feel positively toward the other middle schools in 7 Oaks. Targeting SENSE OF COMMUNITY
11.                        I will recommend Edcamps to others. Targeting VALUE OF EXPERIENCE

 

 

February 15, 2016

The Edcamp was received extremely well and the feedback was overwhelmingly positive. I entered the data into SPSS and experimented with different reports mostly for my own interest to make solid conenctions between my surface understanding of statistics and the data as presented. I understand that the broad but shallow scope of my survey is limiting but it did offer a reasonable test of the questions (some of which need revising) and a sense of the Edcamp's effectiveness at achieveing the goals of PD, PLN development, and idea exchange. I do want to follow up with attendees in a couple of weeks in focus groups to get a sense of the enduring impact or experiences from the event.

February 28, 2016

I've let the project sit for a week or so though it has simmered in my mind during that time. At the moment it feels like I have a ton of information and ideas scattered in different places and it needs sorting, organizing, classifying, refining, and focusing. The other challenge is that this project is serving several purposes: one for my school district to demonstrate the value (if any) of open space events, second for the Edcamp Foundation to offer a questions for other events that assess substantial program goals, third for the 802 Program Inquiry and Evaluation course, and fourth for the 800 Self-Regulated Inquiry and Learning course. Each examines the process and event from a slightly different perspective and crafting responses/content.

I continue to use my chart-paper organizers for broad-scope and long-range planning, module lists for mid-range goals, and sticky notes for daily work goals.

01 long range goals charts
02 midrange goal lists03 immediate goal sticky notes

 

 

 

 

 

 

 

 

Personal assessment of abilities:

Knowledge of Edcamps:

I have organized two Edcamps and attended three others so I am very familiar with the format from the perspective of both planner and participant. I have read several articles by Harrison Owen on Open Space Technology upon which Edcamps are structured. I have also read the Edcamp Foundation book, “The Edcamp Model: Powering Up Professional Learning” and much of the Edcamp Wiki page. One challenge will be separating my experience from my observations and ensuring that my enthusiasm for the PD model

Knowledge of Surveys:

I have not yet created a survey that requires a great a degree of accuracy and precision though I have analysed surveys as a consumer of information, as a researcher and scholar judging a study’s merit, and many years ago as an undergraduate earning extra cash administering telephone surveys for a national market research firm.

Knowledge of research studies

In my previous master’s degree I studies quantitative research and designed, but did not deliver, an experimental study. I have read many hundreds of articles from peer-reviewed journals in the last 4 years and feel prepared to take on this challenge.

 

 

Notes and brainstorming

Kalesse, R. (2014). Teachers lead the way at Edcamps: Participant-driven “unconferences” restore the power of professional development. Reading Today, April/May.

Describes how PD is often a passive experience and the origins of Edcamp. Describes the purpose of Edcamp in giving teachers voice, and seeing it as a tool for the teachers’ own learning

 

Ferriter, W. M., & Provenzano, N. (2013). Self-directed learning for teachers. Phi Delta Kappan, 95(3), 16–21.

Provenzano acknowledges the contributions of social connections to his own innovation and success. Edcamps and social connections bring the trajectory of professional development under the teacher’s control. Offers an example of how Provenzano planned learning experiences with individuals around the globe. The boundaries of what is possible expand as one makes those connections. Using Twitter helps break feedback loops and expose the user to other ideas.Also describes the value of building community even in virtual spaces. Describes some challenges of connections namely balance in one’s life, accuracy of information found, perceived value. These connections let the individual take control of their own professional learning. The ultimate in differentiation for staff learning in contrast to large single message workshops.

 

Carpenter, J. P. (2015). Unconference professional development: Edcamp participant perceptions and motivations for attendance. Professional Development in Education, 42(1), 1–22. http://doi.org/10.1080/19415257.2015.1036303

Good review of the literature relating to professional development, models of delivery, and characteristics of quality PD. Reviews literature on teacher collaboration finding that there is student benefit. Holds Edcamps up to those criteria. Some reference to virtual spaces and ongoing relationships mentioning Gee’s affinity spaces. Studied edcampers’ motivation and perceptions. Suggest that there is little evidence that Edcamps have lasting effect on teacher practice but there are positive influences on teacher attitude and engagement.

 

Darling-Hammond, Linda; McLaughlin, L. (1995). Policies that support professional development in an era of reform. Phi Delta Kappan, 76(8), 597–604.

Effective professional development has a number of characteristics.

  • engages teachers in concrete tasks related to all aspects of learning
  • grounded in processes of inquiry
  • collaborative focus on communities of practice
  • intimately connected to the teachers’ professional work
  • ongoing and intensive collaborative problem-solving
  • It must be connected to other aspects of school change.

Haslam (2010)

  • Participants understanding of the purpose of the PD
  • Ratings of the usefulness or relevance of key components of the PD to current assignments/responsibilities with special attention to teachers’ perceptions of its usefulness in working with their own students
  • perceptions of the extent to which PD met individual professional learning needs
  • ratings of how well the content or focus of the PD aligned with district or school improvement priorities, plans, and goals
  • perceptions of the kinds of support and encouragement teachers received to actively engage in the PD
  • perceptions of the kinds of support and encouragement they received to apply new knowledge and skills in their classrooms
  • ratings of the likelihood of applying new knowledge and skills in the classroom
  • overall ratings of the usefulness of the PD compared with that of other PD

Possible Survey Questions

**Goal is to identify the extent to which edcamp facilitates professional connections and idea flow given that both are characteristics of highly effective teams and contribute to the overall effectiveness of an organization.**

**The research goal is to identify the extent to which the Edcamp experience

 

  • facilitates the exchange of ideas,
  • develops professional networks,
  • promotes professional growth.

 

edcamp facilitates professional connections and idea flow given that both are characteristics of highly effective teams and contribute to the overall effectiveness of an organization.**

 

General Format Questions

  1. What edcamp did you attend?
  2. Was the edcamp you attended part of your school day or on your own time?
  3. Did you make an invitation?
    1. yes: to what extent did the resulting conversation meet your needs?
    2. no: why didn’t you make an invitation?
  4. Do you use social media for professional purposes (facebook, twitter, instagram, pinterest, linkedin, etc.)
  5. Overall, how useful was this day to you in terms of your own professional development?
    1. not very useful o  o  o  o  o very  useful
  6. To what degree does each of the following contribute to meeting your goals for the day? facility, organization, food, social media connections, variety of people attending
  7. Describe the strengths/weaknesses of this kind of professional learning experience?

 

Exchange of Ideas: indicate the degree to which these statements apply to you:

  1. I had topics in mind that I wanted to explore at Edcamp.
  2. I encountered new ideas at Edcamp.
  3. My ideas were heard by others.
  4. I heard others’ ideas.
  5. I was an active participant in session conversations.
  6. I am likely to try something I learned today.

 

Professional Networking

  1. Did you have conversations with people you just met today?
  2. How likely are you to correspond with those people in the next two months?in person, by text/telephone, by email, on social media
  3. To what extent do you feel professionally closer to your colleagues?
  4. In what ways, if any, did your sense of community change?
  5. How frequently do you talk with colleagues in your school about professional topics?
  6. How frequently do you talk with colleagues outside your school about professional topics?
  7. Did you connect with any edcamp participants today using social media?
  8. Did you exchange contact information with another participant today?

 

Professional Growth / Professional Learning

  1. To what extent did you learn - teach at the Edcamp
  2. Did this event contribute to your skills as an educator?
  3. Did this event contribute to your enthusiasm for your job?
  4. On a scale -how motivated were you to attend/participate prior to attending? How about after? (Compare the change in motivation before and after)
  5. were there sessions that addressed topics you were interested in?
  6. What were your goals for the day?
  7. Did you meet your goals?

 

Holding Tank

  1. Did you make an invitation related to your interest?
  2. Did you attend a session related to your interest?

 

References

Fowler, F. J. (1993). Designing questions to be good measures. In Survey Research Methods (2nd ed., pp. 69–93). Newbury Park, CA: Sage Publications Inc. http://doi.org/10.4135/9781452230184O’Brien, J., & Jones, K. (2014). Professional learning or professional development? Or continuing professional learning and development? Changing terminology, policy and practice. Professional Development in Education, 40(5), 683–687. http://doi.org/10.1080/19415257.2014.960688

Bonnie Nardi, Steve Whittaker and Heinrich Schwarz, 2002. “NetWORKers and their activity in intensional networks,” Computer Supported Cooperative Work (CSCW), volume 11, numbers 1–2, pp. 205–242.

Bonnie A. Nardi, Steve Whittaker and Heinrich Schwarz, 2000. “It’s not what you know, it’s who you know: Work in the information age,” First Monday, volume 5, number 5, athttp://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/741/650, accessed 4 May 2011.

M. Bruce Haslam. (2010). Teacher professional development evaluation guide. National Staff Development Council. Washington DC. Retrieved from http://learningforward.org/docs/pdf/evaluationguide.pdf?sfvrsn=0.

O’Brien, J., & Jones, K. (2014). Professional learning or professional development? Or continuing professional learning and development? Changing terminology, policy and practice. Professional Development in Education, 40(5), 683–687. http://doi.org/10.1080/19415257.2014.960688

Pentland, A. (2014). Social physics: How good ideas spread - The lessons from a new science. New York, NY: Penguin Press.

Rajagopal, K., Brinke, D. J., Van Bruggen, J., & Sloep, P. B. (2012). Understanding personal learning networks: Their structure, content and the networking skills needed to optimally use them. First Monday, 17(1), 1–12. http://doi.org/10.5210/fm.v17i1.3559

Vagias, W. (2006). Likert-type scale response anchors. Clemson International Institute for Tourism and …, 3–4. http://doi.org/10.1525/auk.2008.125.1.225

Vuorikari, R., & Scimeca, S. (2013). Social learning analytics to study teachers’ large-scale professional networks. In T. Ley, M. Ruohonen, M. Laanpere, & A. Tatnall (Eds.), Open and Social Technologies for Networked Learning (Vol. 395, pp. 25–34). Springer Berlin Heidelberg. http://doi.org/10.1007/978-3-642-37285-8_3

What do you think? Share you thoughts below...

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: