How to Make ChatGPT Provide Sources | ChatGPT for Students - Software Development PDF Download

Introduction

One of the biggest complaints about ChatGPT is that it provides information, but the accuracy of that information is uncertain. Those complaints exist because ChatGPT doesn't provide sources, footnotes, or links to where it derived information used in its answers.

How to make ChatGPT provide sources and citations

If you know how to properly prompt ChatGPT, it will give you sources. Here's how.

1. Write a query and ask ChatGPT

To start, you need to ask ChatGPT something that needs sources or citations. I've found it's better to ask a question with a longer answer, so there's more "meat" for ChatGPT to chew on. 

Also: The best AI chatbots: ChatGPT and other interesting alternatives to try

Keep in mind that ChatGPT can't provide any information after 2021 and requests about information pre-internet (say, for a paper on Ronald Reagan's presidency) will have far fewer available sources.

Here's an example of a prompt I wrote on a topic that I worked on a lot when I was in grad school:

Describe the learning theories of cognitivism, behaviorism, and constructivism

2. Ask ChatGPT to provide sources

This is where a bit of prompt engineering comes in. A good starting point is with this query:

Please provide sources for the previous answer

I've found that this prompt often provides offline sources, books, papers, etc. The problem with offline sources is you can't check their veracity. But it's a starting point. A better query is this:

Please provide URL sources

This prompt specifically tells ChatGPT that you want clickable links to sources. You can also tweak this up by asking for a specific quantity of sources, although your mileage might vary in terms of how many you get back:

Please provide 10 URL sources

In our next step, we'll see what we can do with these.

3. Push ChatGPT to give you higher-quality sources

Remember that ChatGPT (and most large language model AIs) respond well to detail and specificity. So if you're asking for sources, you can push for higher-quality sources. You'll need to clearly specify you need reliable and accurate sources. While this won't necessarily work, it may remind the AI to give you more useful responses. For example:

Please provide me with reputable sources to support my argument on (whatever the topic is you're looking at)

You can also tell ChatGPT the kinds of sources you want. If you're looking for scholarly articles, peer-reviewed journals, books, or authoritative websites, mention these preferences explicitly. For example:

Please recommend peer-reviewed journals that discuss (and here, repeat what you discussed earlier in your conversation)

Another idea is to use sources that don't have link rot (that is, they're no longer online at the URL ChatGPT might know). Be careful with this, though, because ChatGPT doesn't know of things after 2021. So, while you might be tempted to use a prompt like this:

Please provide me with sources published within the past five years

Instead, consider using a prompt like this:

Please provide sources published from 2019 through 2021.

And, as always, don't assume that whatever ChatGPT gives you is accurate. It's still quite possible it will completely fabricate answers, even to the point of making up the names of what seem like academic journals. It's a fibber. A sometimes helpful tool, but also a fibber.

4. Attempt to verify/validate the provided sources

Keep in mind this golden rule about ChatGPT-provided sources: ChatGPT is more often wrong than right.

Across the many times I've asked ChatGPT for URL sources, roughly half are just plain bad links. Another 25% or more are links that go to topics completely or somewhat unrelated to the one you're trying to source. 

Also: How to use ChatGPT: Everything you need to know

For example, I asked for sources on a backgrounder for the phrase "trust but verify", generally attributed to 1980s US President Ronald Reagan. I got a lot of sources back, but most didn't actually exist. I got some back that correctly took me to active pages on the Reagan Presidential Library site, but where the page topic had nothing to do with the phrase in question.

I had a bit better luck with my learning theory question from step 1. There, I got back offline texts from the people who I knew from my studies had actually worked on those theories. I also got back URLs, but again, only about two in ten actually worked or were accurate.

Also: 40% of workers will have to reskill in the next three years due to AI, says IBM study

But don't despair. The idea isn't to expect ChatGPT to provide sources that you can immediately use. If you instead think of ChatGPT as a research assistant, it will give you some great starting places. Use the names of the articles (which may be completely fake or just not accessible) and drop them into Google. That process will give you some interesting search queries, which will more than likely lead to some interesting reads and material that can legitimately go into your research.

Also, keep in mind that you're not limited to using ChatGPT. Just because ChatGPT exists doesn't mean you should forget all the tools available to researchers and students. Do your own web searches. Check with primary sources and subject matter experts if they're available to you. If you're in school, you can even ask your friendly neighborhood librarian for help.

Also: How to use ChatGPT to create an app

Don't forget that there are many excellent traditional sources. For example, Google Scholar and JSTOR provide access to a wide range of academically acceptable resources you can cite with reasonable confidence.

One final point: if you merely cut and paste ChatGPT sources into whatever research you're doing, you're likely to get stung. Use it for clues, not for a way to avoid the real work of research.

5. Consider upgrading to ChatGPT Plus

While ChatGPT Plus is more expensive (it's $20/month), it has numerous advantages over the free version. In particular, it uses the GPT-4 model, and it adds support for both Code Interpreter and plugins. For our purposes here, the WebPilot plugin can prove very helpful.

Also: How (and why) to subscribe to ChatGPT Plus

WebPilot gives ChatGPT the ability to search the web. This effectively eliminates the 2021 cutoff point for the ChatGPT knowledge base. It also allows ChatGPT to access the web to find more current sources and citations. This has the advantage of reducing the link rot symptoms that come from using a 2021 database in 2023 and beyond.

Also: GPT-3.5 vs GPT-4: Is ChatGPT Plus worth its subscription fee?

Do be aware however, that even with current web access, ChatGPT can still hallucinate. In other words, it will still make stuff up, including valid-seeming sources that don't and have never existed. Still double check everything.