Tag: ChatGPT

Tips: Ensuring Legal and Secure Examinations! Recommendations based on the resource ChatGPT

Posted on by Helena Mörk

How can teachers think and work in higher education to ensure legal and secure examinations? How can we approach the development work from both short-term and long-term perspectives?

Below, you can read a brief discussion on this topic, as well as a compilation of suggestions and recommendations.

Knowledge and insights on how AI, in general, and the AI resource ChatGPT, specifically, can be used and influence us are constantly evolving. New services and applications are being introduced rapidly. However, the fact that ChatGPT has become widely accessible only reinforces our responsibility as course administrators and examiners to design courses that are legally sound and support student learning. In this work, we must constantly ask ourselves: what is the core knowledge in my course, how is it expressed, how do students demonstrate their mastery of that knowledge, and how should we structure the course? In these considerations, open AI can be a resource for both teachers and students, but it also demands that we review questions related to knowledge and legal security.

In the short term, when we have a submitted examination whose authorship we question, the same duty to ensure the origin of the examination applies. This allows us to, for example, have a conversation with the student to verify that they “own their knowledge,” as a supplementary task that can be compared to testing the source in Ouriginal.

To ensure legal and secure examinations, it may be wise to:

  • Test your examinations using Open AI.
  • Conduct examinations in person whenever possible.
  • Design courses with visible processes.
  • Shift from a question-answer format in examinations to a question-(clarified) question format, including justifications.

In the long term, potential development paths could include:

  • Increasing the use of oral examinations and making them digital. You can record them in Zoom, take photographs of a whiteboard with calculations and concept maps, review them afterwards with the student or colleagues. It is also possible to ask the student to prepare a supporting document or present their knowledge in a format that can be saved as evidence for the examination. Oral examinations have the best conditions to be legally sound and can be part of a progressive examination or a culmination of visible learning processes. We know who we are examining, and more students can pass examinations during the course, which is cost-effective. Oral examinations have been proven effective for up to 60-70 students.
  • Designing visible knowledge processes to gain a better understanding of students’ knowledge. Incorporate more practical exercises that do not necessarily need to be graded. Use multiple smaller or progressive examinations and record the results to get closer to our students’ actual knowledge.
  • Clear assessment criteria are crucial for legal security, and this applies to all forms of examination. Develop legally sound seminars that can be combined into examinations and serve as learning opportunities.
  • Shift written examinations from narratives and summaries towards evaluations (including AI-generated responses, perhaps), arguments, identification of false statements, questioning, basing responses on recordings, including concept maps, ranking the significance of concepts, or other graphic representations, and complementing them with examples from the course content. Narratives and summaries can be done in person through various forms of interaction, where it becomes more likely that the student owns their fundamental knowledge.
  • On-campus e-examination.

If you are working with foundational courses or acquiring central knowledge areas, you can ask students in the assessment criteria to make direct connections to the course literature and the topics covered in the course. For advanced courses, you can provide more open-ended instructions for assignments that make it more challenging for AI and encourage students’ independence and critical thinking. They can, to a greater extent, justify and explain their choices and considerations, linking them to the learning objectives.

How to Talk to Your Students about ChatGPT: A Lesson Plan for High School and College Students

Posted on by Karin Thörne

This article by Ben Edwards in ArtsTechnica (April 6, 2023) is worth a read, “Why ChatGPT and Bing Chat are so good at making things up”.

Edwards explains in clear language, with lots of details and examples, how and why large language models (LLMs) such as ChatGPT make up content. As I read this article, it occurred to me that it could serve as a really great way to have pro-active and generative conversations with students about the impact of artificial intelligence for teaching, learning, assessment, and academic integrity. So, here is a quick lesson plan about how to use this article in class:

Education level

Secondary school and post-secondary (e.g., community college, polytechnic, undergraduate or graduate university courses)

Lesson Plan Title: Understanding ChatGPT: Benefits and Limitations

Learning Objectives

By the end of this lesson students will be able to:

  • Understand how and why AI-writing apps make up content.
  • Explain the term “confabulation”.
  • Discuss the implications of fabricated content on academic integrity
  • Generate ideas about how to fact-check AI-generated content to ensure its accuracy

Lesson Preparation

Prior to the class, students should read this article: “Why ChatGPT and Bing Chat are so good at making things up by Ben Edwards, published in ArtsTechnica (April 6, 2023)

Come to class prepared to discuss the article.

Learning Activity

Class discussion (large group format if the class is small or small group format with a large group debrief at the end):

Possible guiding questions:

  • What is your experience with ChatGPT and other AI writing apps?
  • What were the main points in this article? (Alternate phrasing: What were your key takeaways from this article?)
  • What are some of the risks when AI apps engage in confabulation (i.e., fabrication)?
  • Discuss this quotation from the article, “ChatGPT as it is currently designed, is not a reliable source of factual information and cannot be trusted as such.”
  • Fabrication and falsification are commonly included in academic misconduct policies. What do you think the implications are for students and researchers when they write with AI apps?
  • What are some strategies or tips we can use to fact-check text generated by AI apps?
  • What is the importance of prompt-writing when working with AI writing apps?

Duration

The time commitment for the pre-reading will vary from one student to the next. The duration of the learning activity can be adjusted to suit the needs of your class.

  • Students’ pre-reading of the article: 60-minutes or less
  • Learning activity: 45-60 minutes

Lesson closure

Thank students for engaging actively in the discussion and sharing their ideas.

Possible Follow-up Activities

  • Tips for fact-checking. Have students in the class generate their own list of tips to fact-check AI-generated content (e.g., in a shared Google doc or by sharing ideas orally in class that one person inputs into a document on behalf of the class.)
  • Prompt-writing activity. Have students use different prompts to generate content from AI writing apps. Ask them to document each prompt and write down their observations about what worked and what didn’t. Discuss the results as a class.
  • Academic Integrity Policy Treasure Hunt and Discussion. Have students locate the school’s academic misconduct / academic integrity policy. Compare the definitions and categories for academic misconduct in the school’s policies with concepts presented in this article such as confabulation. Have students generate their own ideas about how to uphold the school’s academic integrity policies when using AI apps.

Creative Commons License

This lesson plan is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). This license applies only to the lesson plan, not to the original article by Ben Edwards.

Additional Notes

This is a generic (and imperfect) lesson plan. It can (and probably should) be adapted or personalized depending on the needs of the learners.

Acknowledgements

Thanks to Dr. Rahul Kumar, Brock University for providing an open peer review of this lesson plan.

Share or Tweet this: How to Talk to Your Students about ChatGPT: A Lesson Plan for High School and College Students – https://drsaraheaton.wordpress.com/2023/04/07/how-to-talk-to-your-students-about-chatgpt-a-lesson-plan-for-high-school-and-college-students

10 facts about ChatGPT

Posted on by Cecilia Hellekant

ChatGPT is a revolutionary AI-enabled chatbot that provides long-form answers to complex questions. Trained to learn what humans mean when they ask a question, ChatGPT may be the most impactful tool since the modern search engine.

1. ChatGPT can create content in seconds

ChatGPT is the latest addition to a suite of AI-enabled natural language processors and writing assistants. It was developed and trained on a large dataset of human-to-human conversation transcripts and has been specifically fine-tuned to generate responses in a chatbot context that are generally relevant and appropriate for conversation. Users can ask it a question such as, “What are the five key messages from the work of French philosophers Gilles Deleuze and Felix Guattari?” and the system will generate a written, concise and generally correct response. It can also offer its analysis in several languages. Already a research paper written by ChatGPT – with some editing by Xiaoming Zhai of the University of Georgia – has been published.

Released in November 2022 by OpenAI Foundation, ChatGPT can generate code that can be deployed in software programs. In one case study, it generated complex code in a few minutes — a process that normally takes three to five hours. It can also code in various programming languages. In the example cited, it used Python.

ChatGPT is not connected directly to the Internet and has limited knowledge of the world and events after 2021*. It will make explicit its own limitations and will not generate specific responses to questions where its data sources are either limited or non-existent. For example, asking the system to answer “What is the current weather in Toronto?” will trigger a response that because it can’t browse the Internet it can only provide general information about Toronto weather. It will also challenge incorrect responses to its results.

*A new version was released in January 2023 with updated content up to 2022.

2. ChatGPT is one of a variety of AI-enabled writing tools

There are a growing number of natural language programs that are similar to ChatGPT and can support the writing process. According to the How chatgpt could transform higher education, these include:

  • Writefull – Writefull’s AI helps you write, paraphrase, copy edit and more
  • ecree – A writing tutor
  • TooWrite – A step-by-step guide to writing a paper, with prompts and questions
  • CoAuthor – Human-AI collaborative writing dataset

Other tools that are focused more on corporate copywriting include:

The Washington Post has been using Heliograf to write stories for its business and sports pages since 2016, and later expanded its use for political and economic stories.

3. ChatGPT is currently free to use

Because ChatGPT was developed and deployed by OpenAI, it is free to use. Users need to create a log-in to have access to the program anywhere and at anytime, although access is sometimes restricted because of the high volume of users at certain times.

On its FAQ page, OpenAI makes clear that “free use” is for a research preview phase. There is no indication either of how long this phase will last or what (if any) pricing model may apply once the phase is completed. Other services from OpenAI Foundation, such as the AI-enabled image generator DALL-E, is a pay-per-use service, with monthly free credits available for registered users.

4. ChatGPT learns from its interactions with users.

When ChatGPT is used in a chatbot application, it can continue to learn from the interactions it has with users. This can be done through “fine-tuning,” a process in which the chatbot is fed a new dataset of conversations that are specific to the task or domain the chatbot will be used in. This allows ChatGPT to learn more about the specific language and content that is relevant to the application, and to generate more relevant responses.

One of its main advantages is that it can be easily fine-tuned for specific tasks or domains, allowing it to generate relevant responses to a particular conversation or user need. It is also able to handle a wide range of conversation styles and can generate responses that are appropriate for different types of conversations, including casual, formal and technical.

It’s important to note that ChatGPT is not able to learn in the same way that a human does, but it can continue to improve its performance and generate more appropriate responses through fine-tuning and exposure to new data.

5. ChatGPT (and similar systems) will have a significant impact on programming and code development

A study of reactions to ChatGPT suggests that the most significant near-term impact will be on software development, given the capability of the system to both generate code and to explain how the code it generated works, making software documentation both simpler and quicker.

6. ChatGPT has some limitations

Among the limitations identified since the release of this service are:

  1. Dependence on large amounts of data: ChatGPT, like many other language models, relies on large amounts of data to learn patterns and generate responses. Although this can be an advantage in some cases, it also means ChatGPT may not perform well in tasks or domains where limited data is available.
  2. Lack of common sense: Language models like ChatGPT do not have the ability to understand or use common sense knowledge in the same way humans do. This can lead to responses that are only sometimes appropriate or relevant to the conversation.
  3. Lack of empathy: ChatGPT is not able to understand or respond to emotions in the same way humans do. It may not always be able to generate appropriate responses in emotionally charged situations.
  4. Limited personalization: Although ChatGPT can generate responses that are appropriate for different types of conversations, it is not able to personalize its responses to individual users in the same way a human can.
  5. It is stand-alone: In the launch and research phase, ChatGPT is a stand-alone AI platform not connected to the Internet.

7. ChatGPT doesn’t always get it right.

Because ChatGPT sometimes delivers wrong information, StackOverflow, the go-to question-and-answer site for coders and programmers, banned ChatGPT generated answers. Users were using ChatGPT to flood the site with AI responses, but StackOverflow’s administrators confirmed the answers posted have a high rate of being incorrect. In another example, Dr. Jacqueline Antonovich, an Assistant Professor of History at Muhlenberg College, wrote on Twitter that she put a question from her midterm essays into ChatGPT and the paper it produced “would earn an F. Probably an F – if that’s possible.”

The implication of this for users is that they must develop a critical and analytic process to evaluate the outputs produced by ChatGPT and verify the suggestions it makes.

8. The arrival of ChatGPT is causing some concern for educators

An evaluation of the performance of ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams, was undertaken in December 2022. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Although some see this as a way of supporting students, others are concerned that the quality of ChatGPT responses could lead to a significant increase in cases of academic misconduct, especially for online examinations or take-home essay assignments.

There are other general concerns:

  1. The potential for ChatGPT to reproduce biases and stereotypes that exist in the data it was trained on. Because ChatGPT is trained on a massive amount of text data from the Internet, it is likely to contain biases and stereotypes that are prevalent in society. This can lead to ChatGPT generating responses that are offensive, discriminatory or harmful to certain groups of people. For example, if a student asks ChatGPT a question about a particular group of people, the model may generate a response that is based on stereotypes or prejudices, rather than on accurate or objective information.
  2. The potential for ChatGPT to reduce human interaction and personalized feedback in the learning process. By relying on a machine-learning model to generate responses, students and educators may be less likely to engage directly with each other, which can limit opportunities for meaningful dialogue and the exchange of ideas.
  3. The potential for ChatGPT to create a “one size fits all” approach to learning. When all students receive the same information and feedback based on the model’s output, it can limit their ability to develop their own unique perspectives and voices, and to benefit from personalized instruction or guidance from human educators.
  4. The risk that ChatGPT does not generate accurate or reliable information. Just like Google search results, there is no guarantee that the information ChatGPT provides will be correct or complete. This can lead to students receiving incorrect or misleading information, which can have negative consequences for their learning and understanding.

9. Measures can be taken to reduce the risk of academic misconduct

A writer asked ChatGPT to respond to the question, “What can be done to reduce the risk of ChatGPT being used by students to cheat?” ChatGPT responded with the following:

“To avoid ChatGPT being used for cheating, there are a few different steps that educators and institutions could take. For example, they could:

  • Educate students and educators on the ethical use of AI technology, including ChatGPT, in academic settings.
  • Develop guidelines and policies for the use of ChatGPT in academic work, and make sure that students and educators are aware of and follow these guidelines.
  • Monitor the use of ChatGPT in academic settings and take appropriate action if it is used for cheating or other unethical purposes.
  • Use ChatGPT in ways that support learning and academic achievement rather than as a replacement for traditional forms of assessment. For example, ChatGPT could provide personalized feedback and support to students rather than as a tool for generating entire papers or exams.
  • Incorporate critical thinking and ethical reasoning into the curriculum, to help students develop the skills and habits necessary to use ChatGPT and other AI technology responsibly.”

10. ChatGPT is causing a stir

Since its launch at the end of November 2022, ChatGPT has caused a stir.

A day hardly goes by without a report or opinion piece in popular media about ChatGPT. Twitter is ablaze with comments about its threat, inaccuracies or promises. Even Google reacted with an internal declaration of a “code red” challenge to its search engine. The company is set to announce a series of AI-related developments in May 2023(link is external).

What’s clear at this point is we are only seeing the beginning of the impact of ChatGPT on higher education and our society more generally.

The article is shared with Creative Commons licens

The article 10 facts about chatgpt is copied from Teachonline.ca / Contact North website 2023-02-02, and is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License