pupilfirst logo

Visualising online education alongside AI

This document lays out a roadmap for how we see AI being integrated into the online education landscape in the near future, but mostly as it pertains to Pupilfirst‘s approach to delivering a hands-on learning experience on a variety of topics to students from across the nation, and even across our borders.

This document is organized into different perspectives, namely:

Student

Let‘s start with the role of the Student examining some ways in which AI can reduce student frustration during their learning journey, and enhance it with features not possible through traditional automation.

Adaptive Assistance

One of the key phrases one hears when it comes to the use of AI within education is Adaptive Learning Paths - the idea usually goes along the lines of “an AI can come up with alternate paths of learning for students based on their individual needs”. This idea runs counter to our experience that, at the beginner-level, the concepts that all students need to learn are the same, and the general path (in terms of sequence of concepts to learn) is also the same. We believe that the AI should instead be used to provide Adaptive Assistance, by doing things like:

Enhanced Examples: By providing additional examples and use cases based on where a student is struggling. For instance, if a student a student going through a lesson on IP Addresses and URLs has trouble understanding some part of a lesson, the system can generate more practical examples tailored to their specific misunderstanding.

In-Depth Explanations: Depending on whether the student‘s knowledge of underlying concepts is flawed or missing, offer more detailed explanations or step-by-step guides for complex topics.

Hints and Tips: If a student indicates that they‘re having trouble with an assignment, provide contextual hints and tips, without giving away the solution.

Multilingual Support

Students should be able to converse with their AI assistant in any language they choose - either by typing or simply speaking with the AI - it should be able to reply in their own language, maintain context, and in the case of technical courses, AIs should simply mix technical lingo in the source language (which should be English, for most courses) with the language of student‘s choice.

Multimodal

Often, when students encounter issues, support from subject matter experts require that they “look over the student‘s shoulder” to see what students are seeing. Students may not know what part of a problem they‘re looking at constitutes the important part required for problem solving.

Students should be able to share their screen with the AI assistant so that it can look at something alongside the student - process it the same way that a human at their side would be able to as if they were looking over their shoulder.

In addition to this, students should be able to share pictures such as screenshots, text files such as code or PDFs, links to resources that they may be browsing, or videos that they‘re watching. The AI should be able to process such input and use it to generate responses - as either images, text, or video.

Frequent AI-reviewed assignments

Our experience with human-reviewed assignments that give submissions nuanced grades informs us that students will repeat work, resubmitting to improve grades. This repetition is key to gaining experience through practise. However, we also know that scaling human-reviewed assignments is prohibitively expensive.

AI-assisted generation of assignments is a way for students to quickly check whether their understanding of a topic is up to par.

The AI, observing the response of the student to the assignment, can grade it, converse with students to address deficiencies, and for graded assignments, allow them to take customized assignments.

When a non-maximum grade has been assigned, or when the student requests it specifically, we should allow the AI to generate a brand-new student-specific assignments based on the original stored one, and using the conversation with the student as context, and present it to the student as a way for the student to retake the assignment for improved grades.

The generation of such customized assignments can allow the AI to address student-specific deficiencies, and to target the learning activity on specific weak points.

Personalized progress report

The AI assistant can look at the sum of a student‘s work over time, and give them an overview of what it thinks of their progress. For example, if a student is working irregularly, it can suggest that they stick to workable, but regular schedule of work, and even prepare a study plan for them.

Other examples of what such an expanded report could do include notifying the student if it detects students repeatedly making the same kind of mistake in assignment submissions, asking students to revise certain lessons if it detects weakness in a given area, etc. A similar version of this report should made be available to coaches & TAs and parents.

Do not expect an infallible AI

We should always assume that the AI can, and will make mistakes. It should be possible for students to mark AI responses as incorrect, request regeneration of responses and, when responses are very off the mark, or confusing, students should be able to report it for review.

Students should also be able to easily request help from coaches and teaching assistants from an ongoing conversation with the AI. If human takeover is requested, this should quickly create a Discord forum thread @mentioning the student and TAs and link to a coach-viewable version of the student‘s conversation with AI thus far.

Coach & Teaching Assistant

The role of coaches and student peers who are skilled enough to be teaching assistants cannot be overstated. We need to acknowledge that their time is precious, and that it needs to be deployed efficiently. AI can help us here in many ways.

Course Monitor

Monitoring student activities in a meaningful way is a tedious task that we can heavily automate with the help of AI. When something that needs a student‘s attention is being ignored, the course monitor can step in.

Redirecting the student‘s attention

For example, in an assignment conversation thread, an AI may have given the student a mini-assignment in response to a poor submission, and the student may be ignoring it. The monitor would be able to notice this absence of a response and direct the student to complete the mini-assignment and close out the conversation thread.

Requesting human intervention

The course monitor can converse with coaches and teaching assistants as well.

The monitor may sometimes notice unusual behaviour within the student‘s activity thread. Examples include irrelevant questions being asked of the AI, invalid responses being repeatedly submitted, or simply an unusually long conversation thread about the same topic with the AI. After noticing such unusual behaviour, the monitor should be able to ping coaches and teaching assistants and ask for human intervention.

Monitoring student interactions with AI

It should be made clear to students that interactions with AI are not private. Coaches and teaching assistants should be able to inspect any student interaction with AI. On a student‘s report page, they should be able to see a list of all conversations the student has engaged in with the AI.

Requesting a progress report about the student

When focusing on a student, AI can be used to quickly bring coaches and teaching assistants up to speed about a student‘s progress.

Coaches & teaching assistants should be able to request a learning progress report prepared by the AI by going through all the student‘s artefacts (submissions, timeline of activity, discord messages, etc.). This report should contain a technical report summarizing their status. For coaches and TAs, this could include things like markers of plagiarism, behavioral issues noted, learning deficiencies, regularity of work, etc.

A demo of this process, from the point of view of the teacher, has been included in the relevant section.

Author

A course author‘s role is varied and critical. They act as subject matter experts, defining the contours of the course - its objectives and measurable outcomes - and build and refine its content, alongside defining the tests that measure student progress. AI can assist in this workload in a number of ways, freeing up authors time and allowing them to keep course content up-to-date with ever-increasing standards of work.

AI-enabled content authoring & editing

We need to weave in AI features into course authoring tools; AI can quickly generate starter content based on a given prompt which authors can build on top of, and can also ingest existing content and suggest improvements, based either on its own opinion, or in line with a given prompt.

AI-generated images & videos

In addition to textual content, we should imagine a future where AIs are able to generate images with readable text and stable videos which can accurately portray the requirements given in the prompt. Current AI models cannot do this reliably. However, it isn‘t far-fetched to imagine a near-future with such capabilities.

On-request review of course content

It should be possible for authors and admins to request a full-review of course content. When requested, AIs should be able to load the entire content of the course into context and inspect it for flaws or deficiencies using its latest knowledge. Improvements generated during this process can be presented to authors as a detailed list of suggested changes.

AI-generated & reviewed assignments

We envision AI allowing for an expansion of the quantity and quality of assignments.

AI-assisted assignment creation

Using the course outline, lesson content, and a prompt as context, AI assistants can allow authors to get a quick start on setting up an assignment. Authors can then go through an AI-generated list of questions and approve them or ask the AI to make changes as needed. Once the list of questions looks good, authors can accept the list and save the assignment form.

Multimodal AI review of submissions

Detailed human reviews of student submissions is prohibitively expensive, and prone to errors because of the tedious nature of going through similar submissions. We should expect AI review of assignments to exceed human ability to inspect and critique student submissions, allowing it to tirelessly grade, and provide detailed feedback on student submissions.

Parent / Guardian

We should expect the role of a parent or guardian to be varied, depending on their level of knowledge, and capability to spend time on their children‘s education. Regardless of the depth of their involvement, we should be focused on using AI to present them with information that could be useful in their potential role.

Provide an easy-to-understand overview

Parents should be able to see an overview of their charge‘s progress and to also have the ability to perform limited impersonation; they should be able to view all pages as the student, including assignments submitted and conversations with the AI. They should also be able to see a record of the student‘s posts on Discord.

Requesting a progress report about the student

Parents should also be able to request a learning progress report - information similar to what coaches & TAs are able to request.

Parts of such a detailed report may not make sense to parent, so parents should be allowed to have a conversation with the AI about the report - to clarify aspects of it that they do not understand.

Discord access for parents

Parents and guardians should have access to the Discord server in a read-only manner. Links from the student report should take them to conversation threads in Discord so that they can inspect the context of student conversations. They can also monitor how the student populace is using Discord in general, aiding in expectation-setting.

Teacher

Parents often do not have the time or resources necessary to monitor their children‘s education. In that sense, a teacher is often the stand-in, charged with much the same role. AI allows us to empower teachers with tools that can let them play that role, in a meaningful way, for the large numbers of students whose development they are expected to foster.

Highlighting students who need more attention

Unlike parents, teachers are given charge over a large number of students. Constantly monitoring the needs of a large group can be exhausting. AI assistants such as the Course Monitor do not tire, and can bring up cases of students who need intervention by constantly checking on their activity.

Monitor & mentor

Teachers should be able to request a learning progress report - the same one that coaches & TAs can request.

Similar to parents, teachers should be allowed to have a chat with the AI to understand issues with the student and receive suggestions from the AI on possible methods to address specific issues.

Teachers with subject-matter knowledge can compliment the support given by AI for questions from students and help students understand AI explanation better with other means of interaction (outside LMS and inside a classroom setting) like group discussion, group study, role play, peer-to-peer interactions etc. or even helping students ask better questions to AI.

Teachers can also take help from the AI in planning these in-class activities based on specific assignments or concepts.

Evaluator

While teachers, parents, authors, coaches and teaching assistants play crucial roles in facilitating learning, the ultimate responsibility for engaging with the material, practicing skills, and internalizing knowledge lies with the students themselves.

Regular evaluation is required to ensure that students are meeting their responsibilities and making progress. It is a tool to measure the extent to which students truly learn from the course material they‘re going through, and the assignments they undertake as a part of it.

The best evaluation is a one-on-one conversation between the student and a subject-matter expert. A personal interview, of sorts; AI can help us achieve this with a frequency and level of detail previously inconceivable.

Pupilfirst Trials

We envision a set of closely related tools that allow us to run frequent trials of a student‘s skill. These AI-driven assessments aim to evaluate students‘ understanding and mastery of course material based on detailed expected learning outcomes.

AI-generated questions

The assessment questions are generated by an AI using a rubric document. This rubric outlines expected learning outcomes, grading criteria, and examples of acceptable responses. The AI creates unbiased questions that are closely aligned with the course content without directly replicating it.

Dual AI Interaction

During the test, two AIs will play distinct roles:

Questioner AI: This AI functions as an interviewer, generating and presenting questions one-by-one, ensuring that all expected outcomes are covered. It adapts to the student‘s progress and manages the flow of the assessment.

Reference AI: Acting as a knowledge repository, this AI allows students to query factual information relevant to the ongoing assessment. It provides content-specific answers without directly solving the questions posed by the Questioner AI. The interactions with the Reference AI can influence subsequent questions and the overall assessment result.

Assessment-Driven Teaching

Students will be encouraged to take a formal assessment at the beginning of the course and can retake it after a minimum gap of one month. Assessment results will link directly to course lessons, allowing the Course Monitor AI to guide students on areas needing improvement based on their performance.

Multilingual Testing

To accommodate diverse student backgrounds, assessments will be available in multiple languages, allowing students to take tests in their preferred language.

Regulator

Regulators are charged with making decisions that affect the entire student populace. They rely on access to high-quality data to make informed decisions. We see AI assisting regulators by providing access to comprehensive data sets we possess, including student performance metrics, course completion rates, and feedback on teaching methods.

AI can enhance the interpretation of this data through advanced analytics that identify trends, predict outcomes, and highlight areas requiring attention. Easy access to such data is crucial in enabling regulators to make timely and well-informed decisions that benefit students.

Administrator

In the dynamic and often constrained context of college administration, decision-making spans day-to-day operational concerns such as faculty staffing, issuance of credits, and compliance with educational standards. College administrators also manage funding and navigate the myriad complexities of running an educational institution. AI can significantly alleviate these burdens by automating routine tasks like scheduling and records management, while also serving as a decision support system that offers insights for planning and resource allocation.

Furthermore, AI can act as a vigilant monitor of compliance, ensuring that the institution remains within regulatory guidelines and alerting administrators to potential issues before they escalate.

Hiring Partner

Connecting students to jobs suited to their newly acquired skills is a clear and valuable goal. We aim to build new AI-powered interfaces that allow hiring partners to discuss their requirements, post job listings, and match them to students with the required skills. Partners can also utilize Pupilfirst Trials for conducting additional testing to verify specific skills. These systems can be designed to integrate seamlessly with existing HR tooling, ensuring ease of use and adoption.

Data gathered about job listings and required skills will not only guide students towards in-demand skills but also inform course authors about the need to update or create new content based on industry feedback. Regulators can also gain insights into industry trends, helping them understand workforce dynamics.

This brings us to the final perspective, that of an alumnus:

Alumnus

AI can significantly enhance the engagement and effectiveness of our alumni network. Given the shared focus among our students in deep technical work, AI could enable focused networking, matching alumni based on professional interests, geographic locations, and industry sectors. This networking can be leveraged for career advancement through tailored job alerts, mentorship pairings, and professional development resources, all curated based on the career trajectories and preferences of the alumni.

Additionally, by showcasing the achievements of successful alumni through AI-curated content on various platforms, we can drive existing students to work harder, attract prospective students, and strengthen our standing in the educational community.