{"id":235,"date":"2024-03-27T10:21:35","date_gmt":"2024-03-27T10:21:35","guid":{"rendered":"https:\/\/wp.lancs.ac.uk\/caiss\/?p=235"},"modified":"2024-03-27T10:21:35","modified_gmt":"2024-03-27T10:21:35","slug":"butterflies-and-chatgpt","status":"publish","type":"post","link":"https:\/\/wp.lancs.ac.uk\/caiss\/2024\/03\/27\/butterflies-and-chatgpt\/","title":{"rendered":"Butterflies and ChatGPT"},"content":{"rendered":"\n
Tweet<\/a><\/div>\n
\n
\n
\n
\n

Prompting is the way we talk to generative AI and large language models (LLM\u2019s). The way we construct a prompt can change a models decision on the results it provides and impact the accuracy as well. Research from the University of Southern California Information Sciences Institute shows that a minute tweak – such as a space at the beginning of a prompt can change the results.\u00a0 This is likened to chaos theory where a butterfly flaps its wings generating a minor ripple in the air, resulting in a tornado several weeks later in a faraway land.<\/p>\n

The researchers, who were sponsored by the US Defense Advanced Research Projects Agency (DARPA), chose ChatGPT and applied various different prompt variations.\u00a0 Even slight changes led to significant changes in the results. They found many factors at play and there is more work to be done to ascertain solutions to this effect.<\/p>\n

Why do slight changes result in such significant changes?\u00a0 Do the changes \u201cconfuse\u201d the model?\u00a0 By running experiments across 11 classification tasks, they were able to measure how often the LLM changed its predictions and the impact on accuracy. By studying the correlation between confusion and the instances likelihood of having its answer changed (using a subset of the task with individual human annotations), they did not get a full answer.<\/p>\n

So what?<\/b>:<\/p>\n

Generating LLMs which are resistant to changes and yield consistent, accurate answers is a logical next step.\u00a0 However, this will require a deeper understanding of why responses change under minor tweaks.\u00a0 Is there a way we can anticipate these resulting changes in outputs?\u00a0 With ChatGPT being integrated into systems at scale this work will be important for the future.<\/p>\n

Link to full article.\u00a0<\/a><\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

Prompting is the way we talk to generative AI and large language models (LLM\u2019s). The way we construct a prompt can change a models decision on the results it provides… Continue reading Butterflies and ChatGPT<\/span><\/a><\/p>\n","protected":false},"author":1669,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[3],"tags":[],"class_list":["post-235","post","type-post","status-publish","format-standard","hentry","category-articles","without-featured-image"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/wp.lancs.ac.uk\/caiss\/wp-json\/wp\/v2\/posts\/235","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wp.lancs.ac.uk\/caiss\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wp.lancs.ac.uk\/caiss\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wp.lancs.ac.uk\/caiss\/wp-json\/wp\/v2\/users\/1669"}],"replies":[{"embeddable":true,"href":"https:\/\/wp.lancs.ac.uk\/caiss\/wp-json\/wp\/v2\/comments?post=235"}],"version-history":[{"count":1,"href":"https:\/\/wp.lancs.ac.uk\/caiss\/wp-json\/wp\/v2\/posts\/235\/revisions"}],"predecessor-version":[{"id":236,"href":"https:\/\/wp.lancs.ac.uk\/caiss\/wp-json\/wp\/v2\/posts\/235\/revisions\/236"}],"wp:attachment":[{"href":"https:\/\/wp.lancs.ac.uk\/caiss\/wp-json\/wp\/v2\/media?parent=235"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wp.lancs.ac.uk\/caiss\/wp-json\/wp\/v2\/categories?post=235"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wp.lancs.ac.uk\/caiss\/wp-json\/wp\/v2\/tags?post=235"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}