---
title: "Overview of AI and research tools"
subtitle: "Fall-2024: DSAN AI workshop "
author: "Dr. H"
format: 
    revealjs:
        theme: [default,  ./assets/custom.scss]

---

# A quick overview of AI 

## Data

* Everything starts with data! Once vectorized, data can be used to train AI models
* `Vectorization` $\rightarrow$ representing something as a vector, matrix, or tensor 

`Common types of data`

![](images/2024-09-13-10-25-28.png)


## Example: Image data

* Images are naturally represented as matrices 
* Image processing was a very important phase in the development of modern AI 

![](images/2024-09-13-10-32-48.png)

## Example: Text data

* Text can also be `vectorized` and then used to train LLM models 

![](images/2024-09-13-11-20-55.png){width=700}
   
`Vectorization`:

![](images/2024-09-13-11-21-39.png){width=700}

## What is AI?

* AI is any effort to automate intellectual tasks normally performed by humans
* This is a very broad category; It encompasses all forms of in-organic “intelligence”, e.g. an old calculator technically qualifies as a form of AI

![](images/2024-09-13-10-34-13.png)

* Often the term "AI" these days,refers to some form of `deep learning`

## What is machine learning

![](images/2024-09-13-11-49-52.png)


## Supervised learning

![](images/2024-09-13-11-50-19.png)


## Self-supervised learning 


:::: {.columns}

::: {.column width="45%"}

- Type of unsupervised learning
- Uses input data to generate labels automatically
- No manual labeling required
:::

::: {.column width="55%"}
- Often applied in NLP and vision tasks
- Enables models to learn representations from raw data
- Pretext tasks help train models
:::

::::

![](images/2024-09-13-11-52-17.png)



## What is Deep learning?

* **Deep learning** $\rightarrow$ learning successive layers of increasingly meaningful representations
  * Also known as `layered representation learning` 
    ![](images/2024-09-13-10-40-30.png){width=800}
* Deep learning generally uses a particular `model` known as a `neural network`
    ![](images/2024-09-13-10-45-37.png){width=900}
* This model has many `learnable` fitting parameters


## How does "learning" work?

* `learning` in deep learning is multivariable `optimzation problem` 
* The goal is to `learn` the parameters that `minimize` the model's error on training data 
* The math to train a neural network model are the same as used to train a linear model
  * `Training-data`: $\,y_{data} \,\,\,\,\,$ `model-predictions`:  $\,y_{pred}=y(x)=mx+b\,\,\,\,\,$ 
  * `Loss`: $\, L=(y_{pred}-y_{data})^2\,\,\,\,\,$ `learning`: $\nabla_{m,b}L=\left(\frac{\partial L}{\partial b},\frac{\partial L}{\partial m} \right)=(0,0)$
    ![](https://miro.medium.com/max/1100/0*oqm7QVnI9-inFGCc.gif)

## Generative modeling

:::: {.columns}

::: {.column width="45%"}
- Models data distribution to generate new samples
- Learns from training data patterns
:::

::: {.column width="55%"}
- Can create realistic data (e.g., images, text)
- Examples include GANs, VAEs, and autoregressive models
- Used in creative tasks, simulations, and predictions
:::

::::


![](images/2024-09-13-11-56-45.png)

## Example: This person does not exist 

![](images/2024-09-13-11-57-40.png)

Source: [https://thispersondoesnotexist.com/](https://thispersondoesnotexist.com/)

## Attention mechinism 

* Around 2017, experts started using a new mathmetical mechanism called `attention`
* Attention in a neural network selectively focuses on important input features, enhancing the model’s ability to process complex data.

![](images/2024-09-13-11-45-52.png)

<!-- 
- Mechanism to focus on relevant input parts
- Weighs importance of tokens in context
- Improves context understanding and coherence
- Used in transformers for efficient processing -->


## What are transformers

:::: {.columns}

::: {.column width="45%"}
- Neural network architecture for sequence processing
- Utilizes attention mechanism for context handling
:::

::: {.column width="55%"}
- Efficient for long-range dependencies
- Foundation of large language models (LLMs)
- Processes data in parallel (unlike RNNs)
:::

::::

![](images/2024-09-13-11-47-28.png)

> GPT = `Generative Pre-trained Transformer`


## Scaling laws {.smaller}

- **Efficiency**: Scaling laws help optimize model size, data, and compute resources for better performance.
- **Predictive Power**: They predict how performance improves with more data, larger models, or more computation.
- **Cost-Effectiveness**: Guides investment in resources by showing diminishing returns after certain thresholds.
- **Generalization**: Helps balance model size to avoid overfitting while maximizing language understanding.
- **Benchmarking**: Provides a framework to compare different LLMs and their trade-offs between size and capability.

![](images/image-20240913101402618.png)

## What is an LLM

- **Definition**: A large language model (LLM) is a type of artificial intelligence model designed to understand and generate human language.
- **Architecture**: Typically based on neural networks, especially transformer architectures.
- **Training Data**: Trained on vast amounts of text data from books, articles, websites, etc.
- **Capabilities**: Can perform tasks such as text generation, translation, summarization, question answering, and more.
- **Learning**: Utilizes unsupervised learning, predicting the next word in a sequence based on context.
- **Size**: "Large" refers to the massive number of parameters (billions or more) that allow the model to capture complex language patterns.
- **Applications**: Used in chatbots, virtual assistants, automated content creation, and research.



# Traditional Research tools 

## Example: Research "toolstack"

<!-- * Latex 
* Markdown
* Quarto 
* Zotero  -->

![](images/2024-09-13-12-09-20.png)


## Literature review {.smaller}

`Arguably the most important step of research is the literature review` 

:::: {.columns}

::: {.column width="45%"}
- Summarizes existing research on a topic
- Identifies key themes, gaps, and debates
- Organizes sources logically (chronologically, thematically, etc.)
:::

::: {.column width="55%"}
- Evaluates the credibility of studies
- Provides context for new research
- Highlights trends and areas for further investigation
:::

::::

![](images/2024-09-13-12-20-28.png)

## Citation management {.smaller}

* Example: `.bib` entry  
    ![](images/2024-09-13-12-22-36.png){width=500}
* `Keeping organized is a VERY important part of professional life!`
* It is a good idea to adopt a citation manager early in your academic career
  * https://www.zotero.org/
  * https://www.mendeley.com/

![](images/2024-09-13-12-26-25.png)

# Ai Driven Research tools 

## General comment: Acceleration and efficency  {.smaller}

- **Symbiotic relationship with humans**:
  - AI handles repetitive, time-consuming tasks, freeing humans for creative and strategic work.
  - Enhances human decision-making through advanced data analysis, offering insights for better judgment.
  - Amplifies human capabilities by automating complex calculations, simulations, and predictions.
  - Assists in collaborative problem-solving, blending human intuition with AI's data-driven logic.

- **Efficiency**:
  - AI accelerates workflows by automating routine tasks, reducing human workload.
  - Supports faster information processing, allowing humans to focus on high-level tasks.
  - Enhances productivity by providing real-time recommendations and error detection.
  - Improves human learning and adaptation, enabling quicker mastery of new skills or technologies.

## Mathpix: AI driven OCR {.smaller}

* OCR (Optical Character Recognition) software converts printed or handwritten text from images or PDFs into machine-readable, editable formats.
* Mathpix is an AI-powered tool for converting handwritten or printed content into editable digital formats.
* Ultimatelty it is an `image` to `sequence learning` task

![](images/2024-09-13-12-10-37.png)

Source: [https://mathpix.com/](https://mathpix.com/)

## Text to Voice

A screen reader can significantly boost productivity by enabling multitasking, reducing eye strain, and offering alternative ways to consume written content efficiently. 

*I guarantee that listening to your writing before submission will improve it significantly.*

![](images/2024-09-13-12-13-33.png)

## Image generation 

![](images/2024-09-13-12-21-23.png)

## Isolating relevant journals

![](images/2024-09-13-12-38-51.png)

## Writing with chatGPT {.smaller}


:::: {.columns}

::: {.column width="45%"}
- Good writing communicates clearly and concisely.
- ChatGPT helps improve and speed up writing.
- Always review and refine ChatGPT output for clarity and flow.
:::

::: {.column width="55%"}
- Unedited outputs may lead to dull, robotic text.
- Poor writing doesn't impress or elevate the reader’s opinion.
- ChatGPT has legitimate use-cases when used thoughtfully and edited carefully.
:::

::::

`ALWAYS CHECK WITH YOUR DEPARTMENT FIRST!!!!`

![](images/2024-09-13-12-15-32.png)

`IMPORTANT`: If the text you are inputting to ChatGPT for transformation is NOT ORIGINALLY YOUR OWN content, then you MUST cite the transformed text to provide attribution to the original creator. 

## Summarization with chatGPT {.smaller}

![](images/2024-09-13-12-28-40.png){width=800}

`Example`: 

![](images/2024-09-13-12-30-01.png){width=400}

## chatGPT for project brainstorming 

![](images/2024-09-13-12-33-22.png)

## Going deeper

- **Scholarcy**: Summarizes academic papers, extracting key points, figures, and references for quick review.
- **Scite.ai**: Analyzes citation contexts to assess how research is supported or disputed by other papers.
- **Iris.ai**: Automates literature discovery and connects research papers based on deep content analysis.
- **Research Rabbit**: AI tool for exploring academic networks, finding related research, and tracking new papers in real-time.
- **Semantic Scholar**: AI-powered academic search engine offering enhanced filters, citation tracking, and trend analysis.
- **Connected Papers**: Visual tool leveraging AI to map out the relationships between academic papers, helping users explore relevant literature.
- **ChatGPT**: Assists with brainstorming, summarizing complex topics, and refining research writing.
- **Elicit**: Literature review, systematic review automation.