Overview of AI and research tools
Fall-2024: DSAN AI workshop
A quick overview of AI
Data
- Everything starts with data! Once vectorized, data can be used to train AI models
Vectorization\(\rightarrow\) representing something as a vector, matrix, or tensor
Common types of data

Example: Image data
- Images are naturally represented as matrices
- Image processing was a very important phase in the development of modern AI

Example: Text data
- Text can also be
vectorizedand then used to train LLM models

Vectorization:

What is AI?
- AI is any effort to automate intellectual tasks normally performed by humans
- This is a very broad category; It encompasses all forms of in-organic “intelligence”, e.g. an old calculator technically qualifies as a form of AI

- Often the term “AI” these days,refers to some form of
deep learning
What is machine learning

Supervised learning

Self-supervised learning
- Type of unsupervised learning
- Uses input data to generate labels automatically
- No manual labeling required
- Often applied in NLP and vision tasks
- Enables models to learn representations from raw data
- Pretext tasks help train models

What is Deep learning?
- Deep learning \(\rightarrow\) learning successive layers of increasingly meaningful representations
- Also known as
layered representation learning
- Also known as
- Deep learning generally uses a particular
modelknown as aneural network
- This model has many
learnablefitting parameters
How does “learning” work?
learningin deep learning is multivariableoptimzation problem- The goal is to
learnthe parameters thatminimizethe model’s error on training data - The math to train a neural network model are the same as used to train a linear model
Training-data: \(\,y_{data} \,\,\,\,\,\)model-predictions: \(\,y_{pred}=y(x)=mx+b\,\,\,\,\,\)Loss: \(\, L=(y_{pred}-y_{data})^2\,\,\,\,\,\)learning: \(\nabla_{m,b}L=\left(\frac{\partial L}{\partial b},\frac{\partial L}{\partial m} \right)=(0,0)\)
Generative modeling
- Models data distribution to generate new samples
- Learns from training data patterns
- Can create realistic data (e.g., images, text)
- Examples include GANs, VAEs, and autoregressive models
- Used in creative tasks, simulations, and predictions

Example: This person does not exist

Attention mechinism
- Around 2017, experts started using a new mathmetical mechanism called
attention - Attention in a neural network selectively focuses on important input features, enhancing the model’s ability to process complex data.

What are transformers
- Neural network architecture for sequence processing
- Utilizes attention mechanism for context handling
- Efficient for long-range dependencies
- Foundation of large language models (LLMs)
- Processes data in parallel (unlike RNNs)

GPT =
Generative Pre-trained Transformer
Scaling laws
- Efficiency: Scaling laws help optimize model size, data, and compute resources for better performance.
- Predictive Power: They predict how performance improves with more data, larger models, or more computation.
- Cost-Effectiveness: Guides investment in resources by showing diminishing returns after certain thresholds.
- Generalization: Helps balance model size to avoid overfitting while maximizing language understanding.
- Benchmarking: Provides a framework to compare different LLMs and their trade-offs between size and capability.

What is an LLM
- Definition: A large language model (LLM) is a type of artificial intelligence model designed to understand and generate human language.
- Architecture: Typically based on neural networks, especially transformer architectures.
- Training Data: Trained on vast amounts of text data from books, articles, websites, etc.
- Capabilities: Can perform tasks such as text generation, translation, summarization, question answering, and more.
- Learning: Utilizes unsupervised learning, predicting the next word in a sequence based on context.
- Size: “Large” refers to the massive number of parameters (billions or more) that allow the model to capture complex language patterns.
- Applications: Used in chatbots, virtual assistants, automated content creation, and research.
Traditional Research tools
Example: Research “toolstack”

Literature review
Arguably the most important step of research is the literature review
- Summarizes existing research on a topic
- Identifies key themes, gaps, and debates
- Organizes sources logically (chronologically, thematically, etc.)
- Evaluates the credibility of studies
- Provides context for new research
- Highlights trends and areas for further investigation

Citation management
- Example:
.bibentry

Keeping organized is a VERY important part of professional life!- It is a good idea to adopt a citation manager early in your academic career
- https://www.zotero.org/
- https://www.mendeley.com/

Ai Driven Research tools
General comment: Acceleration and efficency
- Symbiotic relationship with humans:
- AI handles repetitive, time-consuming tasks, freeing humans for creative and strategic work.
- Enhances human decision-making through advanced data analysis, offering insights for better judgment.
- Amplifies human capabilities by automating complex calculations, simulations, and predictions.
- Assists in collaborative problem-solving, blending human intuition with AI’s data-driven logic.
- Efficiency:
- AI accelerates workflows by automating routine tasks, reducing human workload.
- Supports faster information processing, allowing humans to focus on high-level tasks.
- Enhances productivity by providing real-time recommendations and error detection.
- Improves human learning and adaptation, enabling quicker mastery of new skills or technologies.
Mathpix: AI driven OCR
- OCR (Optical Character Recognition) software converts printed or handwritten text from images or PDFs into machine-readable, editable formats.
- Mathpix is an AI-powered tool for converting handwritten or printed content into editable digital formats.
- Ultimatelty it is an
imagetosequence learningtask

Source: https://mathpix.com/
Text to Voice
A screen reader can significantly boost productivity by enabling multitasking, reducing eye strain, and offering alternative ways to consume written content efficiently.
I guarantee that listening to your writing before submission will improve it significantly.

Image generation

Isolating relevant journals

Writing with chatGPT
- Good writing communicates clearly and concisely.
- ChatGPT helps improve and speed up writing.
- Always review and refine ChatGPT output for clarity and flow.
- Unedited outputs may lead to dull, robotic text.
- Poor writing doesn’t impress or elevate the reader’s opinion.
- ChatGPT has legitimate use-cases when used thoughtfully and edited carefully.
ALWAYS CHECK WITH YOUR DEPARTMENT FIRST!!!!

IMPORTANT: If the text you are inputting to ChatGPT for transformation is NOT ORIGINALLY YOUR OWN content, then you MUST cite the transformed text to provide attribution to the original creator.
Summarization with chatGPT

Example:

chatGPT for project brainstorming

Going deeper
- Scholarcy: Summarizes academic papers, extracting key points, figures, and references for quick review.
- Scite.ai: Analyzes citation contexts to assess how research is supported or disputed by other papers.
- Iris.ai: Automates literature discovery and connects research papers based on deep content analysis.
- Research Rabbit: AI tool for exploring academic networks, finding related research, and tracking new papers in real-time.
- Semantic Scholar: AI-powered academic search engine offering enhanced filters, citation tracking, and trend analysis.
- Connected Papers: Visual tool leveraging AI to map out the relationships between academic papers, helping users explore relevant literature.
- ChatGPT: Assists with brainstorming, summarizing complex topics, and refining research writing.
- Elicit: Literature review, systematic review automation.