끄적끄적

설정

Colab 런타임 선택

Gemma 모델을 실행하기에 충분한 리소스가 포함된 Colab 런타임이 필요합니다. 이 경우 T4 GPU를 사용할 수 있습니다.

1. Colab 창 오른쪽 상단에서 ▾(추가 연결 옵션)을 선택합니다.
2. "런타임 유형 변경"을 선택합니다.
3. 하드웨어 가속기에서 T4 GPU를 선택합니다.

 

 

Gemma setup

Gemma 설정의 설정 지침을 완료해야 합니다. Gemma 설정 지침은 다음을 수행하는 방법을 보여줍니다.

• kaggle.com에서 Gemma에 액세스하세요.
• Gemma 2B 모델을 실행하기에 충분한 리소스가 있는 Colab 런타임을 선택하세요.
• Kaggle 사용자 이름과 API 키를 Colab 비밀로 생성하고 구성합니다.

Gemma 설정을 완료한 후 다음 섹션으로 이동하여 Colab 환경에 대한 환경 변수를 설정합니다.

 

 

Kaggle API Key 생성

 

1. Kaggle에 로그인한후 Your Profile 선택

2. Settings 클릭해서 이동

3. API 항목중 Create New Token 클릭

4. 로걸에 다운로드된 kaggle.json 파일에서 username과 key값을 저장합니다.

 

{"username":"dongshik","key":"4a566********************9a7be40"}

 

 

자격 증명(credentials) 구성

Kaggle 자격 증명을 Colab Secrets 관리자에 추가하여 안전하게 저장합니다.

 

1. Google Colab 노트북을 열고 왼쪽 패널에서 🔑 Secrets 탭을 클릭합니다.
2. 새 비밀번호 만들기 (+새 보안 비밀 추가) : KAGGLE_USERNAME 및 KAGGLE_KEY
3. 사용자 이름을 KAGGLE_USERNAME에 복사하여 붙여넣으세요.
4. KAGGLE_KEY에 키를 복사하여 붙여넣으세요.
5. secret 에 대한 "노트북 액세스"를 허용하려면 왼쪽에 있는 버튼을 전환하세요.

 

Python에서 Kaggel Cerdential key 가져오기

import os
from google.colab import userdata

# Note: `userdata.get` is a Colab API. If you're not using Colab, set the env
# vars as appropriate for your system.
os.environ["KAGGLE_USERNAME"] = userdata.get("KAGGLE_USERNAME")
os.environ["KAGGLE_KEY"] = userdata.get("KAGGLE_KEY")

실행했을때 403 Client Error 발생하는경우 kaggle keras gemma 2에 대한 Access 권한이 없기때문입니다.

이럴때 kaggle keara gemma 2로 가서 Access 권한을 요청합니다.

...

You don't have permission to access resource at URL: https://www.kaggle.com/models/keras/gemma2/keras/gemma2_9b_en/2
Please make sure you are authenticated if you are trying to access a private resource or a resource requiring consent.

https://www.kaggle.com/models/keras/gemma2

생략...

M1 교체 이전에 사용하던 MacBook Pro 2015에서 Ollama로 Gemma2를 구동해봤습니다.

Processor: Multi-core CPU (Intel i5/i7/i9 or AMD equivalent)
Memory: You should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.
Storage: At least 20 GB of free space on an SSD
Operating System: Recent versions of Windows, macOS, or Linux

Python 3.11.4로 생성된 Conda 가상환경으로 실행합니다.

(base) Gabrielui-MacBookPro:~ gabriel$ conda env list
# conda environments:
#
base                  *  /usr/local/anaconda3
gabriel                  /usr/local/anaconda3/envs/gabriel

(base) Gabrielui-MacBookPro:~ gabriel$ conda activate gabriel
(gabriel) Gabrielui-MacBookPro:~ gabriel$

(gabriel) Gabrielui-MacBookPro:~ gabriel$ python -V
Python 3.11.4

세 가지 크기: 2B, 9B 및 27B 파라미터의 모델

• 2B Parameters ollama run gemma2:2b
• 9B Parameters ollama run gemma2 (Default)
• 27B Parameters ollama run gemma2:27b

셋중에서 9B 모델이 Default 입니다. 

(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama --version
ollama version is 0.3.9

(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama run gemma2
pulling manifest
pulling ff1d1fc78170... 100% ▕█████████████████████████████████████████████████████████████████▏ 5.4 GB
pulling 109037bec39c... 100% ▕█████████████████████████████████████████████████████████████████▏  136 B
pulling 097a36493f71... 100% ▕█████████████████████████████████████████████████████████████████▏ 8.4 KB
pulling 2490e7468436... 100% ▕█████████████████████████████████████████████████████████████████▏   65 B
pulling 10aa81da732e... 100% ▕█████████████████████████████████████████████████████████████████▏  487 B
verifying sha256 digest
writing manifest
success
>>>

다운로드한 모델을 확인합니다.

(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama list
NAME         	ID          	SIZE  	MODIFIED
gemma2:latest	ff02c3702f32	5.4 GB	14 minutes ago
(gabriel) Gabrielui-MacBookPro:~ gabriel$

Ollama + Gemma2 (9B)

ollama run gemma2 로 다운로드한 모델의 정보를 확인합니다.

>>> /show info
  Model
  	arch            	gemma2
  	parameters      	9.2B
  	quantization    	Q4_0
  	context length  	8192
  	embedding length	3584

  Parameters
  	stop	"<start_of_turn>"
  	stop	"<end_of_turn>"

  License
  	Gemma Terms of Use
  	Last modified: February 21, 2024

>>>

프롬프트로 질문 "why is the sky blue?"

>>> why is the sky blue?
The sky appears blue due to a phenomenon called **Rayleigh scattering**.

Here's a breakdown:

* **Sunlight:** Sunlight is made up of all the colors of the rainbow.
* **Atmosphere:** When sunlight enters Earth's atmosphere, it collides with tiny air molecules (mainly nitrogen and
oxygen).
* **Scattering:** These molecules scatter the sunlight in all directions.
* **Wavelengths:**  Blue light has a shorter wavelength than other colors in the visible spectrum. Shorter wavelengths are
scattered more effectively by the air molecules.

Therefore, blue light gets scattered much more than other colors, making the sky appear blue to our eyes.

**At sunrise and sunset:** The sunlight travels through more atmosphere to reach us. This means even more blue light is
scattered away, allowing longer wavelengths like orange and red to dominate, resulting in those beautiful hues.

>>> Send a message (/? for help)
Use Ctrl + d or /bye to exit.

REST 엔드포인트를 통해 응답 생성

(gabriel) Gabrielui-MacBookPro:~ gabriel$ curl http://localhost:11434/api/generate -d '{
>   "model": "gemma2",
>   "prompt": "Why is the sky blue?"
> }'
{"model":"gemma2","created_at":"2024-09-05T15:30:49.100401Z","response":"The","done":false}
{"model":"gemma2","created_at":"2024-09-05T15:30:49.464952Z","response":" sky","done":false}
{"model":"gemma2","created_at":"2024-09-05T15:30:49.798186Z","response":" appears","done":false}
{"model":"gemma2","created_at":"2024-09-05T15:30:50.143292Z","response":" blue","done":false}



{"model":"gemma2","created_at":"2024-09-05T15:32:05.437344Z","response":" have","done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:05.785728Z","response":" any","done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:06.14108Z","response":" other","done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:06.517148Z","response":" questions","done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:06.926336Z","response":"!","done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:07.269795Z","response":"","done":true,"done_reason":"stop","context":[106,1645,108,4385,603,573,8203,3868,235336,107,108,106,2516,108,651,8203,8149,3868,3402,577,476,27171,3151,5231,15599,44957,38497,168428,235248,109,4858,235303,235256,476,25497,235292,109,235287,5231,219715,66058,175521,603,4247,1644,908,576,832,573,9276,576,573,30088,235265,108,235287,5231,30962,235303,235256,117961,66058,3194,33365,30866,573,10379,235303,235256,13795,235269,665,1041,23524,675,16791,2681,24582,591,80711,23584,578,16175,846,108,235287,5231,102164,574,66058,3766,24582,17109,573,33365,575,832,16759,235265,235248,108,235287,5231,10716,181430,235256,66058,7640,2611,919,476,25270,35571,1178,1156,9276,575,573,12918,18303,235265,10323,576,736,235269,665,6803,30390,1683,978,731,573,2681,24582,1178,5543,95178,1154,3118,689,10436,235265,109,688,2339,235269,1212,783,1443,66058,109,651,30390,3868,2611,26676,1167,4628,774,832,16759,235269,3547,573,8203,4824,3868,235265,2625,48825,578,22097,235269,573,33365,37845,1593,978,576,573,13795,235265,1417,3454,573,3868,2611,603,30390,3024,1693,978,235269,15267,573,5543,95178,591,165117,578,72638,235275,577,6378,1167,4628,235269,10241,1941,4964,9276,235265,110,5331,682,1230,1013,692,791,1089,1156,3920,235341],"total_duration":81320587916,"load_duration":42868270,"prompt_eval_count":15,"prompt_eval_duration":3107284000,"eval_count":210,"eval_duration":78169448000}
(gabriel) Gabrielui-MacBookPro:~ gabriel$

REST 엔드포인트를 통해 Gemma와 채팅

(gabriel) Gabrielui-MacBookPro:~ gabriel$ curl http://localhost:11434/api/chat -d '{
>   "model": "gemma2",
>   "messages": [
>     {
>       "role": "user",
>       "content": "why is the sky blue?"
>     }
>   ]
> }'
{"model":"gemma2","created_at":"2024-09-05T15:32:34.118301Z","message":{"role":"assistant","content":"The"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:34.466319Z","message":{"role":"assistant","content":" sky"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:34.821353Z","message":{"role":"assistant","content":" appears"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:35.182394Z","message":{"role":"assistant","content":" blue"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:35.523289Z","message":{"role":"assistant","content":" due"},"done":false}

{"model":"gemma2","created_at":"2024-09-05T15:33:50.644094Z","message":{"role":"assistant","content":" if"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:33:51.039595Z","message":{"role":"assistant","content":" you"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:33:51.411561Z","message":{"role":"assistant","content":" have"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:33:51.815174Z","message":{"role":"assistant","content":" any"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:33:52.152768Z","message":{"role":"assistant","content":" other"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:33:52.492949Z","message":{"role":"assistant","content":" questions"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:33:52.915517Z","message":{"role":"assistant","content":"!"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:33:53.27064Z","message":{"role":"assistant","content":""},"done_reason":"stop","done":true,"total_duration":80946825634,"load_duration":44007406,"prompt_eval_count":15,"prompt_eval_duration":1749943000,"eval_count":190,"eval_duration":79152305000}
(gabriel) Gabrielui-MacBookPro:~ gabriel$

Ollama + Gemma 1.1 (2B)

(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama run gemma:2b
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling c1864a5eb193... 100% ▕██████████████████████████████████████████████████████████████████████████████████▏ 1.7 GB
pulling 097a36493f71... 100% ▕██████████████████████████████████████████████████████████████████████████████████▏ 8.4 KB
pulling 109037bec39c... 100% ▕██████████████████████████████████████████████████████████████████████████████████▏  136 B
pulling 22a838ceb7fb... 100% ▕██████████████████████████████████████████████████████████████████████████████████▏   84 B
pulling 887433b89a90... 100% ▕██████████████████████████████████████████████████████████████████████████████████▏  483 B
verifying sha256 digest
writing manifest
success
>>> Send a message (/? for help)

다운로드한 모델을 확인합니다.

(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama list
NAME         	ID          	SIZE  	MODIFIED
gemma:2b     	b50d6c999e59	1.7 GB	3 minutes ago
gemma2:latest	ff02c3702f32	5.4 GB	40 minutes ago
(gabriel) Gabrielui-MacBookPro:~ gabriel$

Gemma2와 동일한 질문을 합니다. "why is the sky blue?"

>>> why is the sky blue?
The sky appears blue due to Rayleigh scattering.

**Rayleigh scattering** is a scattering phenomenon where light waves are scattered in different directions depending on their wavelength.
Blue light has a shorter wavelength than other colors of light, meaning it scatters more effectively. This is why the sky appears blue.

Here's a more detailed explanation:

* **Sunlight** contains all colors of the rainbow.
* **Blue light** has a shorter wavelength than other colors.
* **When blue light** interacts with molecules in the atmosphere, it **scatters more strongly** than other colors.
* This scattering causes blue light to be scattered **towards the observer**.
* The scattered blue light is what we see in the sky as **blue color**.

**Other factors that contribute to the blue color of the sky:**

* **Dust particles:** Air molecules can also scatter light, but their scattering effect is generally weaker than that of molecules in the
atmosphere.
* **Water droplets:** Water droplets in the atmosphere can also scatter light, but their effect is typically less significant than that of
molecules.
* **Sun's position:** The sky appears blue because the sun is lower in the sky, meaning it is directly overhead. This means that more blue
light reaches our eyes.

In summary, the blue color of the sky is due to Rayleigh scattering of light by molecules in the atmosphere, which primarily results from
the shorter wavelength of blue light.

>>> Send a message (/? for help)

실행시에 MacBook의 성능을 확인합니다.

실행한 모델 정보 확인

>>> /show info
  Model
  	arch            	gemma
  	parameters      	3B
  	quantization    	Q4_0
  	context length  	8192
  	embedding length	2048

  Parameters
  	repeat_penalty	1
  	stop          	"<start_of_turn>"
  	stop          	"<end_of_turn>"

  License
  	Gemma Terms of Use
  	Last modified: February 21, 2024

>>> Send a message (/? for help)

Ollama + Gemma 1.1 (7B)

이번엔 Gemma 7B 모델을다운받아 테스트합니다.

(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama run gemma:7b
pulling manifest
pulling ef311de6af9d...  74% ▕█████████████████████████████████████████
pulling manifest
pulling ef311de6af9d...  74% ▕█████████████████████████████████████████
pulling manifest
pulling ef311de6af9d...  74% ▕█████████████████████████████████████████
pulling manifest
pulling ef311de6af9d... 100% ▕███████████████████████████████████████████████████████████████▏ 5.0 GB
pulling 097a36493f71... 100% ▕███████████████████████████████████████████████████████████████▏ 8.4 KB
pulling 109037bec39c... 100% ▕███████████████████████████████████████████████████████████████▏  136 B
pulling 65bb16cf5983... 100% ▕███████████████████████████████████████████████████████████████▏  109 B
pulling 0c2a5137eb3c... 100% ▕███████████████████████████████████████████████████████████████▏  483 B
verifying sha256 digest
writing manifest
success
>>> Send a message (/? for help)

다운로드한 모델 확인

(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama list
NAME             ID              SIZE      MODIFIED
gemma:7b         a72c7f4d0a15    5.0 GB    2 weeks ago
gemma:2b         b50d6c999e59    1.7 GB    2 weeks ago
gemma2:latest    ff02c3702f32    5.4 GB    2 weeks ago

Gemma2와 동일한 질문을 합니다. "why is the sky blue?"

>>> why is the sky blue?
**The sky is blue due to a phenomenon called Rayleigh scattering.**

* Sunlight is composed of all the colors of the rainbow, each with a different wavelength.
* Shorter wavelengths of light, such as blue light, have higher energy and more tightly spaced photons.
* When sunlight interacts with molecules in the atmosphere, such as nitrogen and oxygen, the shorter wavelengths are
scattered more effectively.

**Here's how it works:**

1. Sunlight enters the atmosphere and interacts with molecules of nitrogen and oxygen.
2. These molecules have electrons that can be excited by the energy of the incoming light.
3. The electrons absorb the energy and vibrate, then emit it in all directions.
4. However, **blue light is scattered more efficiently than other colors** because its shorter wavelength allows for more
frequent collisions between the photons and the molecules.

**This scattering process results in:**

* More blue light is scattered in all directions, reaching our eyes and making the sky appear blue.
* Longer wavelengths of light, such as red light, are scattered less efficiently, so we see more of them directly from
the sun.

**Additional factors influencing the sky's color:**

* **Time of day:** The sky is generally bluer at midday when the sunlight travels through less atmosphere.
* **Cloud cover:** Clouds can block the sunlight and scatter less blue light, making the sky appear less blue.
* **Pollution:** Pollution in the atmosphere can also scatter light and affect the sky's color.

>>> Send a message (/? for help)

실행한 모델 정보 확인

>>> /show info
  Model
  	arch            	gemma
  	parameters      	9B
  	quantization    	Q4_0
  	context length  	8192
  	embedding length	3072

  Parameters
  	penalize_newline	false
  	repeat_penalty  	1
  	stop            	"<start_of_turn>"
  	stop            	"<end_of_turn>"

  License
  	Gemma Terms of Use
  	Last modified: February 21, 2024

>>> Send a message (/? for help)

질의 성능 확인

Intel CPU의 MacBook에서 Gemma2 9B 모델 그리고 Gemma 2B, 7B 모델까지 Ollama로 구동해 봤습니다.

감사합니다.