2013 Logo 3d gradient 
SEGway        September 2014
Research and Assessment News from SEG Measurement  
In This Issue


Follow us on Twitter


View our profile on LinkedIn


View our videos on YouTube


Like us on Facebook



Find us on Google+



31 Pheasant Run

New HopePennsylvania 18938
(267) 759-0617

Welcome to the September issue of SEGway.  September marks the beginning of SEG's tenth year anniversary celebration (and my 37th year conducting educational research and developing educational assessments).  It has been such a treat to have helped more than 60 organizations demonstrate product effectiveness and develop high quality assessments.

We are all carefully watching the implementation of Common Core and other new state standards and the implementation of the PARCC and Smarter Balanced Assessments.  It should come as no surprise that we are seeing a significant uptick in publishers and tech providers asking SEG for help conducting effectiveness research and developing assessments for products aligned to the new CCSS.

We hope you find this issue of SEGway useful; let us know what you think.  In our "Technical Corner" we share the news about the recent release of the new Standards for Educational and Psychological Testing. SEG Measurement endorses the Standards and we use them to guide our research and assessment work.

The impending birth of the "royal baby" presents an opportunity to explain a common fallacy in understanding probability.

SEG has recently begun several new assessment and research projects and we have shared one of those with you.  We are seeing a lot more interest in educational games, so we are sharing some of the work we are doing with GoNoodle, Inc. in assessing the effectiveness of their Mega Math Marathon educational game.

We are heading out to the EdNET Conference next week and then on to EDUCAUSE, NJEA and AACE and are looking forward to seeing many of you there. Please let us know in advance if you are attending any of these so we can meet and learn more about your work in the education marketplace. 

Also, take a look at our website at www.segmeasurement.com, as it is continually updated with developments in the field.  And, feel free to email me at selliot@segmeasurement.com.  I'd love to hear what's on your mind.

 

Sincerely,

 

Scott Signature  

 

 

Scott Elliot

 

SEG Measurement

 

Five Tips for Recruiting Efficacy Study Participants

It should come as no surprise that school buyers are looking for hard evidence of product effectiveness.  With more and more educational products for schools to choose from and limited resources, buyers want to be sure they are spending their money well.

One of the biggest challenges faced when conducting efficacy research is recruiting participants.   Since a good controlled study includes both users of the product under study and those not using the product, this can be particularly tough.

After more than three decades of conducting studies, we have learned a thing or two about this at SEG.  Here are five tips to make this task a bit easier and more effective.  Contact us if you want to know more about recruitment--or if you want to learn more about what is involved in conducting efficacy research.

  1.  Do the work up front to invite the most targeted potential participants. Make sure that you know ahead of time the type of schools or classes that are needed for the study.  Then be sure your recruitment is targeting folks who meet those criteria.  Sounds simple, but so much recruitment time and effort is wasted on reaching out to folks who should not be contacted.  Sometimes it may feel easier to do a widespread recruitment, but the extra work to conduct targeted recruitment is worth the effort.  Consider current clients that may be a good fit.  Ask for referrals from participants as they express interest.  Include prequalification questions in early contact so that folks can determine whether they qualify and may be interested. 
  2. Spread the word through many channels and make it easy for participants to find you.   Now that you have a targeted plan for recruitment, be sure to use many channels.  Potential participants are not going to seek you out, they need to be recruited.  Include emails, calls, social media, and direct mail.  And, make it very easy for interested participants to contact you.  If you are hard to reach or there are too many steps, potential participants will give up. 
  3. Highlight the reasons for the study and what will be done with the results.  This involves generating a sense of urgency and interest for folks to want to be involved in something so interesting and important.  In other words, if no one cares or shares an interest in the outcomes, it is going to be very difficult to arouse interest in potential participants.
  4. Show that you value the participants' time and energy.  In other words, be sure to incent the participants with a reasonable compensation for their time and effort in terms of product, money, or other resources.  Be sure to optimize the value to peak interest while also not making it so lucrative that folks are participating simply for the incentive. 
  5. Address and minimize any potential concerns or roadblocks to participation.  We all know that most schools and teachers are not excitedly waiting for the next research project.  Participation takes work and some dedication for success.  So, it is important that the responsibilities of the participants are made clear up front.  Clarify how data will be protected.  Explain how the time and inconvenience will be minimized.  Support any approval and review processes that are needed for each school.
     
When you target the right folks, share a compelling message about the project, spread the word widely, incent folks appropriately, show that you value everyone's time, and help to remove roadblocks, you will be well on your way to study recruitment success!  And as your participant list grows, do not forget to over-recruit to accommodate the attrition that is expected to occur during the school year.   

Click here for more about efficacy research and sample studies 

 

SEG has worked with many educational publishers and technology providers, from start-ups to the largest industry players, to design & conduct efficacy research programs.

 

With nearly 40 years of experience in research, we know what it takes to conduct sound efficacy research.  Please contact us to discuss your needs.  Email us or call us at 800.254 7670 ext. 102.

Mega Math Marathon Game: New Tool for Improving Math Skills
Growing interest in efficacy of educational games

GoNoodle, Inc. launched a new physical activity platform, Mega Math Marathon.  Since launching in August 2013, the company has seen explosive growth with over 100,000 teachers and 2,000,000 kids actively engaged each month. Mega Math Marathon gets K-8 kids up and moving with physical activity breaks during the classroom day.  General research supports the connection between increased physical activity and academic achievement. Anecdotal feedback from teachers, using the education-aligned games as a supplemental resource for core subjects, is that it is improving student performance and test scores.

 

GoNoodle, Inc. wanted to assure current users and potential users with scientific evidence of its effectiveness.  So, they asked SEG to conduct a controlled study of product efficacy.  We are  studying schools throughout the country using a quasi-experimental design.  Simply put, we are comparing math skill gains for a group of teachers and students using Mega Math Marathon to a comparable group of nonusers.  Both groups will receive a pre and posttest of mathematics skills and they will be compared statistically.  Teacher perceptions of effectiveness will also be explored.

 

Let us know if we can help you gather the evidence you need to help convince your customers that your product is effective by emailing us at info@segmeasurement.com.
Measurement Moment
The Royal Baby: What are the odds?
Unless you have been living under a rock, it is highly likely (pun intended) that you know that Prince William and Princess Kate are having another child.  This not only gives us an opportunity to come clean about our secret English Royal obsession with you, it is also a great opportunity to take a look at the common mistakes people make with regard to probabilities.

   

The UK and others throughout the world are wagering furiously on everything from whether Kate and William will have a boy or a girl to the name that will be chosen. A recent article said that 68% of those betting are betting the royal baby will be a girl. Why is the betting running heavily in favor of having a girl despite that fact that we know there is about a 50/50 chance of having a boy or girl?  The most likely reason that having a girl is so heavily favored is a lack of understanding of the concept of statistical independence.

 

There is a good chance (pun intended) that bettors are running heavily in favor of having a girl is because the first royal baby was a boy.  Common (but faulty) logic says that if Kate and William already had a boy, they are more likely to have a girl.  This has the same roots as the common gambler's fallacy of being "due"--that belief that simply because you have lost 10 hands in a row, you are "due" for a winning hand. 

 

The problem with this logic is that having a child is an "independent event."  Each birth is independent of the others--what happened with the last child (or the last hand) does not affect what happens next.  Each birth, or hand, is unaffected by the other births or hands. Even if you have previously given birth to a girl, the probability of the next child being a boy or girl remains at 50/50.  So, if you are wagering on the royal birth, do not fall victim to the fallacy that a girl is "due."

 


This illustrates the basic principle of statistical independence; but, before I receive the complaint letters, let me acknowledge that the actual birthrate actually runs at about 51%, or as high as 52% boys in some countries/cultures.  I also acknowledge that there are other factors that may affect the sex of the baby that suggest that this is not purely independent. 

 

Technical Corner
New Standards for Educational and Psychological Testing Released
business_people.jpg
The American Educational Research Association, American Psychological Association and the National Council on Measurement recently published the new Standards for Educational and Psychological Testing. 

Why should you care? SEG Measurement and our colleagues involved in assessment strive to provide valid, reliable and fair tests that provide the information educators need to make informed decisions. The Standards represent the collective wisdom of the field of assessment and provide the guidance to achieve that goal.

If you are involved in assessment in any way--whether you administer high stakes tests or just include assessments in your products, these standards should guide your work. Much like Electricians or Engineers have standards to which they adhere to ensure that the work you get from them is up to snuff and safe, test developers and providers have standards to guide their practice.

The Standards provide guidance for developing valid, reliable and fair tests.  They also provide important information to guide test design, scoring, scaling, setting cut scores, reporting results and documentation.  The Standards also address the rights and responsibilities of test takers and test users, and provide guidance for specific testing applications such as psychological testing, educational testing and credentialing.

You can get the Standards (at the lowest cost) directly from the American Educational Research Association (AERA).

SEG Measurement endorses the Standards and is committed to providing valid, reliable and fair assessments.  Contact us at 800-254-7670 or on the web at www.segmeasurement.com.
Are you Contemplating an Effectiveness Study for Your Product?
Attend Our Roundtable Session at EdNET

Evaluating the Effectiveness of Digital Learning Tools and Educational Games: What can we learn from the experience of industry veterans?

  

The number of digital tools and educational games being offered is increasing dramatically.  But, school buyers are understandably skeptical. They want evidence that the tools they are using increase student learning, engagement and critical thinking. According to a KnowledgeQuest study, more than four out of five school buyers want to see proof of product effectiveness before purchasing. Testimonials and case studies are no longer enough to satisfy buyers.

  

Many digital tool and game providers are seeking scientific support for the efficacy of their products. Those contemplating conducting efficacy research have many questions.  Fortunately, there is much to be learned from the experience of others in the industry.

This session brings together those who have conducted research with those who are at the early stages.  Recent studies from Cengage, Imagine Learning, Kindermusik and HealthTeacher.com will be discussed and analyzed to get helpful tips and understand potential pitfalls and challenges.

Moderator: Scott Elliot, President, SEG Measurement  

Tuesday, September 30, 9:30 AM



About SEG Measurement
medical-team-portrait.jpg

SEG Measurement conducts technically sound product efficacy research for educational p
ublishers, technology providers, government agencies and other educational organizations, and helps organizations build better assessments. We have been meeting the research and assessment needs of organizations since 1979. SEG Measurement is located in New Hope, Pennsylvania and can be accessed on the web at

 SEG At Upcoming Conferences
Let's Meet!

We were pleased to see many of our colleagues at Head Start's 12th National Research Conference.  Interest in educational products and services at the district and state level are increasing and we are seeing a wave of new innovations. We look forward to seeing you at the upcoming conferences we will be attending. 

  • EdNET 2014 Bronze Sponsor, September 28-30, Baltimore, MD
    • Business Opportunities Fair, September 29 at 5pm
    • Birds-of-a-Feather Roundtable: Digital Learning Tools & Educational Games, September 30 at 9:30am
  • EDUCAUSE Annual Conference, September 29-October 2, Orlando, FL
  • AACE E-Learn 2014 World Conference on E-Learning, October 27-30, New Orleans, LA
    • Effectiveness of Digital Learning Tools on Student Achievement in Higher Education, October 28 at 1:30pm
    • A Study of the Effectiveness of an e-Learning Application on Student Learning in Higher Education Calculus I Courses, To Be Scheduled

If you would like to meet with a representative from SEG Measurement to discuss how we might help you with your assessment and research needs, please contact us at info@segmeasurement.com.

In Memoriam
Terry Jankowski
On September 19, 2014, Terry Jankowski, a long time employee and friend to all of us at SEG Measurement passed away.  She will be missed, but we have faith that she will be working with God on the Heaven Admissions Testing Program.