{"id":616,"date":"2025-09-01T15:13:04","date_gmt":"2025-09-01T15:13:59","guid":{"rendered":"https:\/\/www.curtin.edu.au\/assessment2030\/assessment-design-studio\/reflexive-essay-2\/"},"modified":"2025-09-12T01:43:25","modified_gmt":"2025-09-12T01:43:25","slug":"critical-ai-comparison-report","status":"publish","type":"page","link":"https:\/\/www.curtin.edu.au\/assessment2030\/assessment-design-studio\/critical-ai-comparison-report\/","title":{"rendered":"Critical AI Comparison Report"},"content":{"rendered":"\n<h3 class=\"wp-block-heading\">Overview<\/h3>\n\n\n\n<p>Critical AI Comparison Reports are intended to serve as mechanisms for students to consider how AI tools perform in comparison human-led outputs. The goal in this is for students to develop evaluative judgement in the context of AI and hone their own responsible approach to AI for their future professional practice.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Key features<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Lane 2: Non-secure assessment<\/li>\n\n\n\n<li>Supported by both in-person and online teaching modes<\/li>\n\n\n\n<li>Scope for students to participate as individuals, pairs or teams<\/li>\n\n\n\n<li>Encourages students to build confidence in their own analytical capabilities before exploring AI tool contributions<\/li>\n\n\n\n<li>Develops professional digital literacy through hands-on experimentation with various AI platforms and approaches<\/li>\n\n\n\n<li>Builds evaluative judgement by requiring students to critically assess AI outputs against their own reasoning<\/li>\n\n\n\n<li>Prepares students for responsible AI integration in their future professional contexts<\/li>\n\n\n\n<li>Promotes understanding of AI tool diversity and the importance of strategic prompting<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">How it works<\/h3>\n\n\n<mimas-block \n        block-name=\"accordion\"\n        role=\"complementary\"\n        aria-label=\"Accordion content\"\n        data-segment=\"all\">\n\n                \n\n                    <div class=\"accordion\" aria-labelledby=\"accordion-title-Set-up-\">\n            <button type=\"button\" class=\"accordion__heading\" aria-controls=\"Set-up-\" aria-expanded=\"false\">\n            <div class=\"title\" id=\"accordion-title-Set-up-\">Set-up <\/div>\n            <div class=\"icon\"><\/div>\n        <\/button>\n        <div class=\"accordion__content\" id=\"Set-up-\" aria-hidden=\"true\">\n                    <ul>\n<li>Educators provide datasets, case studies, or scenarios suitable for analysis using unit concepts<\/li>\n<li>Students are encouraged to approach the analytical challenge initially through human reasoning and disciplinary knowledge \u2013 and that this will help aid their later reflection<\/li>\n<li>Curriculum and in-class time are used to provide initial support on AI tool experimentation, including guidance on testing different platforms and prompting approaches<\/li>\n<li>Assessment framework emphasises learning through comparison and developing professional AI literacy<\/li>\n<\/ul>\n\n        \n                            \n                    <\/div>\n<\/div>\n                    <div class=\"accordion\" aria-labelledby=\"accordion-title-Progression\">\n            <button type=\"button\" class=\"accordion__heading\" aria-controls=\"Progression\" aria-expanded=\"false\">\n            <div class=\"title\" id=\"accordion-title-Progression\">Progression<\/div>\n            <div class=\"icon\"><\/div>\n        <\/button>\n        <div class=\"accordion__content\" id=\"Progression\" aria-hidden=\"true\">\n                    <ul>\n<li>Students begin analysis using their own disciplinary knowledge and reasoning skills (ideally without AI assistance)<\/li>\n<li>Students then use a range of AI tools to analyse the same materials, documenting prompts and outputs to understand the range of AI capabilities<\/li>\n<li>Students systematically compare findings, identifying where AI succeeded, failed, or provided different perspectives<\/li>\n<li>Students critically evaluate findings, considering where AI insights have complemented or challenged their human analysis<\/li>\n<li>Regular check-ins help students share in a peer network their reflections on appropriate AI integration for their future professional practices<\/li>\n<\/ul>\n\n        \n                            \n                    <\/div>\n<\/div>\n                    <div class=\"accordion\" aria-labelledby=\"accordion-title-Submission\">\n            <button type=\"button\" class=\"accordion__heading\" aria-controls=\"Submission\" aria-expanded=\"false\">\n            <div class=\"title\" id=\"accordion-title-Submission\">Submission<\/div>\n            <div class=\"icon\"><\/div>\n        <\/button>\n        <div class=\"accordion__content\" id=\"Submission\" aria-hidden=\"true\">\n                    <ul>\n<li>Students submit comprehensive reports documenting their initial human-led analysis, as well as later AI exploration, critical comparison of various approaches and professional insights about the responsible use of AI<\/li>\n<li>Documentation also includes examples of students AI \u2018fails\u2019 as well as AI \u2018wins\u2019 demonstrating their understanding of AI diversity and strategic application<\/li>\n<\/ul>\n<p><span data-ccp-props=\"{&quot;134233117&quot;:true,&quot;134233118&quot;:true,&quot;201341983&quot;:0,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\n\n        \n                            \n                    <\/div>\n<\/div>\n        \n<\/mimas-block>\n\n\n<h3 class=\"wp-block-heading\"><strong>Curtin snapshot&nbsp; &nbsp;<\/strong><\/h3>\n\n\n    <mimas-block \n        block-name=\"articles\"\n        role=\"complementary\"\n        aria-label=\"articles content\"\n        data-segment=\"all\"\n        data-bg=\"gray\">\n\n        \n\n                    <div class=\"article\">\n                                                                \n                <img decoding=\"async\" src=\"https:\/\/www.curtin.edu.au\/assessment2030\/wp-content\/uploads\/2025\/09\/JoseLoureiro-150x150.jpg\"\n                     alt=\"Jose Loureiro\"\n                     srcset=\"https:\/\/www.curtin.edu.au\/assessment2030\/wp-content\/uploads\/2025\/09\/JoseLoureiro-150x150.jpg 150w, https:\/\/www.curtin.edu.au\/assessment2030\/wp-content\/uploads\/2025\/09\/JoseLoureiro-740x370.jpg 740w, https:\/\/www.curtin.edu.au\/assessment2030\/wp-content\/uploads\/2025\/09\/JoseLoureiro-768x768.jpg 768w, https:\/\/www.curtin.edu.au\/assessment2030\/wp-content\/uploads\/2025\/09\/JoseLoureiro-800x500.jpg 800w, https:\/\/www.curtin.edu.au\/assessment2030\/wp-content\/uploads\/2025\/09\/JoseLoureiro.jpg 800w, https:\/\/www.curtin.edu.au\/assessment2030\/wp-content\/uploads\/2025\/09\/JoseLoureiro.jpg 800w, https:\/\/www.curtin.edu.au\/assessment2030\/wp-content\/uploads\/2025\/09\/JoseLoureiro-480x240.jpg 480w, https:\/\/www.curtin.edu.au\/assessment2030\/wp-content\/uploads\/2025\/09\/JoseLoureiro-800x630.jpg 800w, https:\/\/www.curtin.edu.au\/assessment2030\/wp-content\/uploads\/2025\/09\/JoseLoureiro-800x760.jpg 800w\"\n                     sizes=\"(min-width: 2560px) 2560px, (min-width: 1920px) 1920px, (min-width: 1440px) 1440px, (min-width: 1024px) 1024px, (min-width: 768px) 768px\" loading=\"lazy\">\n\n                <article class=\"article__content\">\n                                            <mimas-eyebrow>Case Study <\/mimas-eyebrow>\n                    \n                                            <h2 class=\"h3\">Dr Jose Loureiro<\/h2>\n                    \n                                            <p><p>\u201c<span class=\"NormalTextRun SCXW176005295 BCX0\">While AI tools like ChatGPT can quickly provide insightful and data-driven suggestions, it&#8217;s valuable for <\/span><span class=\"NormalTextRun SCXW176005295 BCX0\">students \u2013 our future investors <\/span><span class=\"NormalTextRun SCXW176005295 BCX0\">\u2013 <\/span><span class=\"NormalTextRun SCXW176005295 BCX0\">to<\/span><span class=\"NormalTextRun SCXW176005295 BCX0\"> understand the underlying assumptions and calculation methods<\/span><span class=\"NormalTextRun SCXW197374470 BCX0\">.<\/span>\u201d<\/p>\n<p><strong><span class=\"TextRun SCXW124140557 BCX0\" lang=\"EN-AU\" xml:lang=\"EN-AU\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW124140557 BCX0\">Faculty of Business and Law <\/span><\/span><span class=\"EOP SCXW124140557 BCX0\" data-ccp-props=\"{&quot;134245418&quot;:true,&quot;134245529&quot;:true}\">\u00a0<\/span><\/strong><\/p>\n<\/p>\n                    \n                                \n                        <\/article>\n            <\/div>\n        <\/mimas-block>\n\n\n\n<h3 class=\"wp-block-heading\">Jose&#8217;s example assessment<\/h3>\n\n\n\n<p><strong>About my unit:<\/strong> Faculty of Business and Law&nbsp;| Above 1000 students | Hybrid | Group work<\/p>\n\n\n\n<p>I redesigned the analytical report assessment in ECOM1000 to help students develop both financial analysis skills and AI literacy simultaneously. Student groups receive two investment portfolios and must determine which offers better returns using Modern Portfolio Theory and Sharpe Ratio calculations.<\/p>\n\n\n\n<p>The assessment follows a strict sequence: groups first complete their own calculations and analysis without AI assistance, documenting their methodology and reasoning. Only after submitting their independent analysis do, they use AI tools to analyse the same portfolios. Groups then critically compare the approaches, identifying where AI provided valuable insights, made errors, or used different assumptions.<\/p>\n\n\n\n<p>This comparative approach revealed fascinating learning opportunities. For example, one group discovered that AI made calculation errors but provided useful context about market conditions they hadn&#8217;t considered. Another group found that AI used different risk assumptions, leading them to better understand the importance of clearly defining parameters in financial modelling.<\/p>\n\n\n\n<p>The assessment culminates in groups presenting refined recommendations that integrate the strongest elements from both their original analysis and AI insights while clearly articulating the limitations they identified in AI outputs.<\/p>\n\n\n\n<p><strong>My advice<\/strong>&nbsp;<\/p>\n\n\n\n<p>Encourage students to complete independent work without AI first &#8211; this is crucial for developing their own analytical capabilities before AI comparison. Provide standard prompts for initial AI interactions to ensure consistency, but require students to experiment with follow-up questions.<\/p>\n\n\n\n<p>Focus marking on student reasoning and critical evaluation rather than the quality of AI outputs. The learning happens in the comparison and synthesis phases, where students must defend their methodology and integrate insights. I also recommend requiring students to identify specific examples where they trusted their analysis over AI recommendations and explain their reasoning.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Suggested marking criteria<\/h3>\n\n\n<mimas-block \n        block-name=\"accordion\"\n        role=\"complementary\"\n        aria-label=\"Accordion content\"\n        data-segment=\"all\">\n\n                \n\n                    <div class=\"accordion\" aria-labelledby=\"accordion-title-AI-exploration-and-systematic-experimentation-(25%)\">\n            <button type=\"button\" class=\"accordion__heading\" aria-controls=\"AI-exploration-and-systematic-experimentation-(25%)\" aria-expanded=\"false\">\n            <div class=\"title\" id=\"accordion-title-AI-exploration-and-systematic-experimentation-(25%)\">AI exploration and systematic experimentation (25%)<\/div>\n            <div class=\"icon\"><\/div>\n        <\/button>\n        <div class=\"accordion__content\" id=\"AI-exploration-and-systematic-experimentation-(25%)\" aria-hidden=\"true\">\n                    <p>Shows comprehensive exploration of AI tool capabilities through varied platforms, prompting strategies, and approaches. Documents experimentation process effectively, demonstrating understanding of how different tools and prompts produce different outputs.<\/p>\n\n        \n                            \n                    <\/div>\n<\/div>\n                    <div class=\"accordion\" aria-labelledby=\"accordion-title-Critical-evaluation-and-comparison-(40%)-\">\n            <button type=\"button\" class=\"accordion__heading\" aria-controls=\"Critical-evaluation-and-comparison-(40%)-\" aria-expanded=\"false\">\n            <div class=\"title\" id=\"accordion-title-Critical-evaluation-and-comparison-(40%)-\">Critical evaluation and comparison (40%) <\/div>\n            <div class=\"icon\"><\/div>\n        <\/button>\n        <div class=\"accordion__content\" id=\"Critical-evaluation-and-comparison-(40%)-\" aria-hidden=\"true\">\n                    <p>Demonstrates sophisticated analysis of AI tool strengths, limitations, and inconsistencies. Shows strong evaluative judgement in identifying where AI provides valuable insights versus where it produces errors, biases, or superficial responses. Articulates clear understanding of AI capabilities across different analytical contexts.<\/p>\n\n        \n                            \n                    <\/div>\n<\/div>\n                    <div class=\"accordion\" aria-labelledby=\"accordion-title-Professional-AI-literacy-and-insight-(25%)-\">\n            <button type=\"button\" class=\"accordion__heading\" aria-controls=\"Professional-AI-literacy-and-insight-(25%)-\" aria-expanded=\"false\">\n            <div class=\"title\" id=\"accordion-title-Professional-AI-literacy-and-insight-(25%)-\">Professional AI literacy and insight (25%) <\/div>\n            <div class=\"icon\"><\/div>\n        <\/button>\n        <div class=\"accordion__content\" id=\"Professional-AI-literacy-and-insight-(25%)-\" aria-hidden=\"true\">\n                    <p>Develops nuanced understanding of when and how to integrate AI tools responsibly in professional practice. Shows ability to make informed decisions about AI use, demonstrates awareness of ethical considerations, and articulates sophisticated approach to AI integration that maintains analytical integrity.<\/p>\n\n        \n                            \n                    <\/div>\n<\/div>\n                    <div class=\"accordion\" aria-labelledby=\"accordion-title-Communication-and-documentation-(10%)\">\n            <button type=\"button\" class=\"accordion__heading\" aria-controls=\"Communication-and-documentation-(10%)\" aria-expanded=\"false\">\n            <div class=\"title\" id=\"accordion-title-Communication-and-documentation-(10%)\">Communication and documentation (10%)<\/div>\n            <div class=\"icon\"><\/div>\n        <\/button>\n        <div class=\"accordion__content\" id=\"Communication-and-documentation-(10%)\" aria-hidden=\"true\">\n                    <p>Clearly presents comparative analysis and experimental findings. Effective documentation of AI exploration process enables understanding of methodology. Professional presentation of complex comparative insights and decision-making frameworks.<\/p>\n\n        \n                            \n                    <\/div>\n<\/div>\n        \n<\/mimas-block>\n\n\n<p>Note: Marking criteria and weighting are suggested guidelines. Specific descriptions should be adapted to relevant content and learning objectives.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Overview Critical AI Comparison Reports are intended to serve as mechanisms for students to consider [&hellip;]<\/p>\n","protected":false},"author":9,"featured_media":0,"parent":161,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":true,"_oasis_is_in_workflow":0,"_oasis_original":0,"_oasis_task_priority":"","footnotes":""},"tags":[],"class_list":["post-616","page","type-page","status-publish","hentry"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.curtin.edu.au\/assessment2030\/wp-json\/wp\/v2\/pages\/616","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.curtin.edu.au\/assessment2030\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.curtin.edu.au\/assessment2030\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.curtin.edu.au\/assessment2030\/wp-json\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/www.curtin.edu.au\/assessment2030\/wp-json\/wp\/v2\/comments?post=616"}],"version-history":[{"count":0,"href":"https:\/\/www.curtin.edu.au\/assessment2030\/wp-json\/wp\/v2\/pages\/616\/revisions"}],"up":[{"embeddable":true,"href":"https:\/\/www.curtin.edu.au\/assessment2030\/wp-json\/wp\/v2\/pages\/161"}],"wp:attachment":[{"href":"https:\/\/www.curtin.edu.au\/assessment2030\/wp-json\/wp\/v2\/media?parent=616"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.curtin.edu.au\/assessment2030\/wp-json\/wp\/v2\/tags?post=616"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}