In most modern LLMs different tools are already available to users extending the functionality of these models often beyond the core Language functionality. Some of the possible tools you might encounter are:
- Web searches
- Most modern LLMs do web searches by default if they think the information they need for the response is too recent for their initial training dataset. Some suppliers offer web search as an option that you can turn on and off. Most allow you to do the same using the prompt by saying something like: Do not use web search.
- Deep Thinking
- Introduced in the present generation of LLMs is a feature that forces the LLM to go over it’s first response and look at it from another angle. This often leads to better responses and by looking at the thinking you can understand how the LLM reached its conclusion. This offers a form of accountability of the LLM!
- Document uploading
- Most LLMs offer the possibility to upload one or more document in the current Context Window. This allows the LLM access to this new information and subsequent queries will take this information into account. This means you can have the LLM perform actions just on your documents.
- App generation
- Many LLMs allow for the automatic generation of online applications following your prompts. This creates small web-based applications completely written by the LLM that can often be saved and shared. One such example is the Artifact feature of Claude.
- Speech interaction
- Although often only available via mobile phone apps, some models allow for speech input and speech output. Some do not use Speech-To-Text but use the actual audio as tokens and generate audio tokens as well, forgoing the translation to and from text. This can be called True Audio or Advanced Speech processing like with ChatGPT.
- Image use & generation
- You can upload images with text that the LLM can read and add to its information so you can question its contents. Other tools like Dall-E3 are special AI models, sometimes integrated into LLM, for image generation based on prompts.
- Video interaction
- Some models (like ChatGPT), often only via mobile phone apps, allow for live streaming video input that the AI model will use as information. You can ask live questions (often via speech) about the content of the video.
- Video generation
- Most video generation AI models do not function from within LLMs so are not strictly tools that can be used in an LLM interface. There are many suppliers of video generation tools like Open AI Sora, Google Veo2, etc. Note: you will need a paid subscription to use these tools.
