We asked ChatGPT to write an article about Penn State’s capabilities in AI, from materials science to ethics and sustainability. And then, we asked David Hunter, who was recently appointed as director for a cross-functional Penn State AI Hub, for his reaction to what artificial intelligence said about the University.
“I accept your challenge!” he said. “But first a digression. A friend of mine asked ChatGPT to write a press release about Penn State’s AI Hub and the results were ... interesting. It showed me a use for ChatGPT I hadn’t previously thought of – suggesting things to specifically avoid. I guess this makes sense; to the extent that a neural network trained on a bunch of information currently publicly available can describe your entity, your entity is not novel. Put a different way, if ChatGPT writes it, you should make sure you really mean it before you write it too.”
“Remember, there’s nothing magical about this program; it’s simply very good at matching a particular set of inputs to an appropriate set of outputs, using as its 'matching' rule a truly enormous body of human knowledge,” he added. “There are several implications of this, not least that you only get from ChatGPT an answer that is as good as what can be found in that body of knowledge.”
Credit: Mike Fleck / Penn State. Creative Commons
ChatGPT: As an AI language model, I can provide information about AI in general and Penn State's initiatives related to AI, but I don't have access to real-time information or specific details about current capabilities of AI at Penn State beyond what has been publicly shared.
DH: The above passage is interesting. I have very little ChatGPT experience, but most of what we’ve been reading about it doesn’t talk about passages like this. It’s almost as though its creators have explicitly added a feature to try to educate users about what it is and is not—seems very responsible, actually!
There’s nothing magical about this program; it’s simply very good at matching a particular set of inputs to an appropriate set of outputs, using as its "matching" rule a truly enormous body of human knowledge.
ChatGPT: That being said, Penn State has been actively working to advance AI research and education across a variety of fields, from healthcare to sustainability. Some of the AI initiatives and capabilities at Penn State include:
Overall, Penn State is actively working to advance the state of AI research and education and to apply AI to solve real-world problems in a variety of fields.
DH: On one hand, the list is amusing: It’s not even close to what a human familiar with Penn State would put together, since it’s so all over the place it doesn’t really give a coherent snapshot of AI-related activity at Penn State.
The list is amusing: It’s not even close to what a human familiar with Penn State would put together.
On the other hand, there are some interesting aspects, like its mention of sustainability. I’d be curious to see what it would find if one were to ask ChatGPT do a deep dive into the sustainability. Obviously all of the same information would be available by simply searching the web, but ChatGPT will perform the rote search and compile the results for you. In this way, tools like ChatGPT represent nothing more than a democratization of access to computing power that has existed for a long time.
What are some of the initiatives happening with AI at Penn State?
ChatGPT: Penn State has several ongoing initiatives related to AI. Here are some examples:
DH: ...some examples of the high variability in the quality of items in lists generated by ChatGPT, that is. Some of these items are useful, while some are ridiculous. Specific comments below:
DH: I have no idea what this is! I guess we can add it to the list of “things that ChatGPT turns out that don’t jibe with reality.”
ChatGPT:
DH: Maybe this is going on at Penn State, and I’d like to learn more if so; but a quick search doesn’t turn up anything specific to Penn State.
ChatGPT:
DH: This is legit. Definitely something to make sure to highlight somehow.
ChatGPT is a helpful tool for automating the process of compiling “to do” items. Yet like any tool, it clearly requires human interaction to be effective.
ChatGPT:
DH: The College of Medicine is an “initiative”? Way too broad to be helpful.
ChatGPT:
DH: Like #3 , the MPS in AI is something to highlight.
Chat GPT: These are just a few examples of the many AI-related initiatives happening at Penn State. The university is committed to advancing AI research and education and to applying AI to solve real-world problems in a variety of fields.
DH: As with the earlier list, ChatGPT is a helpful tool for automating the process of compiling “to do” items. Yet like any tool, it clearly requires human interaction to be effective.
ChatGPT: An AI Hub at Penn State would provide a central hub for AI research and education across the university, allowing researchers from various disciplines to collaborate more easily and efficiently. Here are some ways an AI Hub could unite AI at Penn State:
Overall, an AI Hub at Penn State could help unite AI research and education across the university, providing a central hub for interdisciplinary collaborations, access to resources, education and training, and outreach and engagement.
DH: Nothing wrong with any of this! That said, it would not be helpful to use this text on a public-facing website. Almost by definition, whatever is generated by ChatGPT is not truly original. Technically, it’s “new” in the sense that these precise words have never been combined in exactly this way, but it’s all merely based on existing ideas. If there is anything novel about the AI Hub, it will not be captured by ChatGPT using a query like this.
ChatGPT: As an AI language model, I can search for information and publications related to Penn State AI. However, I have a knowledge cutoff of September 2021, and I do not have real-time access to the latest publications or news.
DH: This is a really interesting preamble; the message feels like something ChatGPT has been programmed to insert as a way of teaching the humans that use it what it can and cannot reasonably be expected to help with. It’s sort of like an “intelligent instruction manual” that knows when to feature relevant passages.
I’m not sure whether to put more faith in a ChatGPT-generated list of publications related to AI at Penn State than, say, a Google search.
ChatGPT: That being said, here are a few examples of recent mentions of Penn State AI in publications:
It's important to note that these examples are not an exhaustive list, and there may be other recent publications or mentions of Penn State AI that I am not aware of.
DH: #2 is flat wrong: ICDS was not launched in 2021 in any sense. There is a brief history of the Institute here, and as you’ll see, that claim about ICDS and its history is wrong on a number of levels!
We all know at some level that that last sentence is true for any web search we ever perform, and there’s nothing wrong with a tool that reminds us of its limitations.
I’ll add here that I’m not sure whether to put more faith in a ChatGPT-generated list of publications related to AI at Penn State than, say, a Google search. In the end, ChatGPT is clearly a useful tool but one that needs supervision when it’s being asked to aid in a creative process. (This is distinct from, say, a task that is well-defined, like driving a car on roads, where a well-designed modern computer program is FAR better than humans at the same task.)