Community Board> Ask QM

Printer-friendly version

This is a place to ask your questions about online and hybrid course quality, particularly as it relates to Quality Matters. 

Please post as a COMMENT and make your Subject as specific as possible.  QM will be monitoring and responding, but others are encouraged to respond as well.  Because Quality Matters is a large Community of Practice, your views are central to what we can accomplish.

Submitted by dadair on

Although there is no one-size-fits-all answer to this question, the Quality Matters Program provides a Course Format Chart that identifies differences between online, blended, and F2F courses.  Within the category of blended courses, Quality Matters further describes two different types depending on whether more or less than 50% of the course occurs in the F2F environment.

Even in the case where more than 50% of the course occurs F2F, Quality Matters requires that the following material be available online if the course is to be evaluated as a quality blended course:

All content, materials, activities, handouts, support materials, etc. included in the face-to-face meetings should also be made available in the online classroom or via an electronic course portfolio. Notable exceptions to this include drama, movie, speech, and lab courses.

Submitted by dadair on

With the initial editions of the QM Rubric, Dr. Ron Legon, QM Executive Director and Provost Emeritus from University of Baltimore, prepared the following document to map the QM Rubric standards with accreditation standards "Comparison of the Quality Matters Rubric to Accreditation Standards in Distance Learning."

For a copy of this document, please email Grace Hall

Good morning.

What is the stipend paid to external reviewers for an institutionally-managed official review? Are there any other costs to the institutions associated with these reviews?

Thanks!

Laurie Posey, GWU

Submitted by dadair on

Laurie,

QM pays $150 to reviewers and $350 to external reviewers. This is the compensation for QM-managed reviews; we don't dictate the fees paid in subscriber-managed reviews.  This relationship is between the institution and the reviewers. We expected the duties of the reviewers to be the same in both types of reviews; however, QM does ask Master Reviewers to select their review team in QM-managed reviews.  This task is typically is handled by IRs in subscriber-managed reviewers. A number of institution, therefore, pay Master Reviewers $250.

Submitted by mpoole on

Just wanted to add a clarification for stipend payments.  For QM managed reviews, QM pays the chair of the review $350 and the other two reviewers $150.

Melissa Poole

Quality Matters Course Review Manager

Submitted by Deb Adair on

HI Lilia

The cost of a QM-managed course review for subscribers is $1000 per course.  For non-subscribers, it is $1400.

Submitted by ebachmann on

We are exploring the use of internal review teams (using the new rubric) and want to get a feel for what time committment that might be expected for review team members.  We are most interested in how much time the Master Reviewer or review team chair must commit, but would also be interested to know how many hours other team members spend, including the developer.  If you can point me to research where that has been quantified, I'd appreciate it!


Regards,


Ed (ebachmann@pierce.ctc.edu | 253-964-6377)

Edward A. Bachmann
Director, eLearning
Pierce College District - 9401 Farwest Dr SW
Lakewood, WA 98498
http://www.pierce.ctc.edu/el

Hi Edward, I spoke with the course review team and here is the response to your question on time spent on reviews.

We base the time from our surveys and we see the responses to be:

How many hours, including conferences,did you spend on the course review process?
 
1. 3-5 hours  
21%
2. 5-7 hours  
28%
3. 7-10 hours  
31%
4. 10+ hours  
20%
Total
100%

The MR will spend about 1-2 more than the other two reviewers.  Hope this helps.

In the process of designing a course, there has been disagreement on how to meet QM expectations regarding assignment instructions for assignments submitted to a dropbox.

One perspective is that the dropbox instructions should be with the dropbox and not elsewhere to avoid students having to navigate to another area is the course to get the instructions.  If the instructions are long and involved, the instructions can be attached to the dropbox as a downloadable file.

Another perspective is that it is not intuitive to have students go to a dropbox file and download it to get instructions for an assignment, and that instructions should be elsewhere in the course content pages (even if these can be duplicated as a downloadable file in the dropbox). 

Considering standards 1.1, 1.2, 2.4 and 3.3, what is the QM perspective on instructions of this sort?

-----

This is related to a larger question of meeting QM standards through a variety of organizational and navigational methods--as many decisions will be based upon interpretations of the standards (and how particular elements might meet these), are there any additional guidelines to aid institutions with the interpreting process for decisions of this kind?

Thank you very much!

Hi Daniel,

Thanks for your question. Providing clear instructions with the assignment and in the drop-box makes sense to me, however, I haven't seen your overall course. It is impossible for me to say that your course would meet standards since each team would make that decision for themselves, and this is just one small design decision that you've asked about.

You aslo asked about additional guidelines. Please look to the annotations in the QM Rubric as your guide and authority in making these kinds of decisions. I would also recommend taking the "Designing Your Online Course" workshop to help work through the standards.

I hope this helps answer your question. Please ask again if it does not.

Best regards,

Brenda

We are currently in a process to review LMS systems at our college.  In looking at impact, one of the questions that came up is whether changing LMS systems would impact those courses that have been certified through the QM review process within the last year.  We 'just' started reviewing courses and putting them through the peer review process and wanted to be 100% that we knew the answer to this question.

I vaguely remember seeing it documented somewhere the percentage of the course that has to change before it needs to go through the peer review process again but assuming that nothing other than the format/technology of the course changes can we assume our courses will still be certified if we change systems?

Please advice.  Thanks!

 

- Melanie

Melanie,

Using your scenario that nothing but the LMS would change as a basis, please see the response below.

  • LMS/CMS changes would not violate the certification as long as major functionality is maintained.  If a new LMS removed student interaction, or had major navigation problems, it might jeopardize QM certification, but this is highly unlikely since schools generally change LMS/CMS platforms to upgrade functionality and reliability.  If the opposite were to occur, it would affect all courses at the institution.

Hi Mary Jo,

Internal (informal) reviews are conducted depending on the goals of the institution.  We do recommend that those serving on internal reviews have at least taken the Applying the QM Rubric so that they are familiar with the rubric use.  From experience, I see most institutions mirroring the official course review procedures.  They use three reviewers and use a subject matter expert.  Because this review is informal they cannot use an external reviewer.  You will need someone on the team to serve as the chair which allows them to submit the final report and approve amendments if needed.  I would be happy to set up a time to talk about the process.  You can email me at mpoole@qualitymatters.org.

We have a question from some people we would like to put through the peer reviewer training.  Requirement #2 for being a Peer Reviewer states: "Current online teaching experience of a for-credit course (within the last 18 months)."

- Will a hybrid class satisfy that requirement?
- Is that a requirement to take the peer reviewer training, or just to become a peer reviewer?

 

 

Manager, Instructional Design and Technology
Wichita State University

John,

A blended/hybrid course does count towards the eligibility.  Anyone can take the Peer Reviewer Course.  We have many Instructional Designers that will also take the PRC even though they are ineligible to become a peer reviewer.

Standard 8.2 looks at third party accessibility. What about the accessibility of the LMS when the course does not use a third party vendor? Is a statement for accessibility of the LMS necessary?

Thank you!

Thanks for your question. Standard 8.2 doesn't look specifically at third-party accessibility, but for information regarding required technologies in the course, including the LMS, regardless of the origin of the LMS, be it proprietary or third party. Many institutions do use third party technologies. Standard 8.2 is looking for statements about the degree to which any required technologies are accessible.

Does this help to answer your question? If not, please ask again.

Brenda Boyd

Director of Professional Development & Consulting
Co-Chair of the Fifth Edition, 2014 Rubric Committee

Would it violate QM policy to place original materials purchased from QM, such as the APPQMR training folder, on physical reserve in the university library as a resource that might be checked out by faculty or the administration?  This material would be treated like any other resource that is copyright protected.

Hello All,

I had a question on Standard 2.4.  Hopefully someone out there can help.  Standard 2.4 states that the relationship between learning objectives and course activities is clearly stated and gives examples such as a numbering system or a narrative explaining how the course activities enable learners to meet objectives. We work with different groups and are several different designers here that review courses.  Originally, some of our courses had placed the statement, " You will meet the objective listed above through a combination of the following activities in this course: Complete all readings, written assignments, quizzes, and final exam. Participate in discussion and peer review assignments."  In your opinion, would this be sufficient to meet this standard or would you mark this as not met?  We've had internal and external reviewers and some have come back as Met and some as Not Met, so I am just wondering what Quality Matters might think about this statement.  Thanks in advance!

Rita

In my opinion, Specific Review Standard 2.4 is not met with a simple statement that learners will complete assignments to meet the objectives. Specific Review Standard 2.4 was overhauled significantly in the Fifth Edition to focus on the relationship between the objectives and the other components of alignment in a course.

I will continue my reply with extensive Annotation citations privately.

Best,

Brenda Boyd