M1 교체 이전에 사용하던 MacBook Pro 2015에서 Ollama로 Gemma2를 구동해봤습니다.
Processor: Multi-core CPU (Intel i5/i7/i9 or AMD equivalent)
Memory: You should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.
Storage: At least 20 GB of free space on an SSD
Operating System: Recent versions of Windows, macOS, or Linux
Python 3.11.4로 생성된 Conda 가상환경으로 실행합니다.
(base) Gabrielui-MacBookPro:~ gabriel$ conda env list
# conda environments:
#
base * /usr/local/anaconda3
gabriel /usr/local/anaconda3/envs/gabriel
(base) Gabrielui-MacBookPro:~ gabriel$ conda activate gabriel
(gabriel) Gabrielui-MacBookPro:~ gabriel$
(gabriel) Gabrielui-MacBookPro:~ gabriel$ python -V
Python 3.11.4
세 가지 크기: 2B, 9B 및 27B 파라미터의 모델
- 2B Parameters ollama run gemma2:2b
- 9B Parameters ollama run gemma2 (Default)
- 27B Parameters ollama run gemma2:27b
셋중에서 9B 모델이 Default 입니다.
(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama --version
ollama version is 0.3.9
(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama run gemma2
pulling manifest
pulling ff1d1fc78170... 100% ▕█████████████████████████████████████████████████████████████████▏ 5.4 GB
pulling 109037bec39c... 100% ▕█████████████████████████████████████████████████████████████████▏ 136 B
pulling 097a36493f71... 100% ▕█████████████████████████████████████████████████████████████████▏ 8.4 KB
pulling 2490e7468436... 100% ▕█████████████████████████████████████████████████████████████████▏ 65 B
pulling 10aa81da732e... 100% ▕█████████████████████████████████████████████████████████████████▏ 487 B
verifying sha256 digest
writing manifest
success
>>>
다운로드한 모델을 확인합니다.
(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama list
NAME ID SIZE MODIFIED
gemma2:latest ff02c3702f32 5.4 GB 14 minutes ago
(gabriel) Gabrielui-MacBookPro:~ gabriel$
Ollama + Gemma2 (9B)
ollama run gemma2 로 다운로드한 모델의 정보를 확인합니다.
>>> /show info
Model
arch gemma2
parameters 9.2B
quantization Q4_0
context length 8192
embedding length 3584
Parameters
stop "<start_of_turn>"
stop "<end_of_turn>"
License
Gemma Terms of Use
Last modified: February 21, 2024
>>>
프롬프트로 질문 "why is the sky blue?"
>>> why is the sky blue?
The sky appears blue due to a phenomenon called **Rayleigh scattering**.
Here's a breakdown:
* **Sunlight:** Sunlight is made up of all the colors of the rainbow.
* **Atmosphere:** When sunlight enters Earth's atmosphere, it collides with tiny air molecules (mainly nitrogen and
oxygen).
* **Scattering:** These molecules scatter the sunlight in all directions.
* **Wavelengths:** Blue light has a shorter wavelength than other colors in the visible spectrum. Shorter wavelengths are
scattered more effectively by the air molecules.
Therefore, blue light gets scattered much more than other colors, making the sky appear blue to our eyes.
**At sunrise and sunset:** The sunlight travels through more atmosphere to reach us. This means even more blue light is
scattered away, allowing longer wavelengths like orange and red to dominate, resulting in those beautiful hues.
>>> Send a message (/? for help)
Use Ctrl + d or /bye to exit.
REST 엔드포인트를 통해 응답 생성
(gabriel) Gabrielui-MacBookPro:~ gabriel$ curl http://localhost:11434/api/generate -d '{
> "model": "gemma2",
> "prompt": "Why is the sky blue?"
> }'
{"model":"gemma2","created_at":"2024-09-05T15:30:49.100401Z","response":"The","done":false}
{"model":"gemma2","created_at":"2024-09-05T15:30:49.464952Z","response":" sky","done":false}
{"model":"gemma2","created_at":"2024-09-05T15:30:49.798186Z","response":" appears","done":false}
{"model":"gemma2","created_at":"2024-09-05T15:30:50.143292Z","response":" blue","done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:05.437344Z","response":" have","done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:05.785728Z","response":" any","done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:06.14108Z","response":" other","done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:06.517148Z","response":" questions","done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:06.926336Z","response":"!","done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:07.269795Z","response":"","done":true,"done_reason":"stop","context":[106,1645,108,4385,603,573,8203,3868,235336,107,108,106,2516,108,651,8203,8149,3868,3402,577,476,27171,3151,5231,15599,44957,38497,168428,235248,109,4858,235303,235256,476,25497,235292,109,235287,5231,219715,66058,175521,603,4247,1644,908,576,832,573,9276,576,573,30088,235265,108,235287,5231,30962,235303,235256,117961,66058,3194,33365,30866,573,10379,235303,235256,13795,235269,665,1041,23524,675,16791,2681,24582,591,80711,23584,578,16175,846,108,235287,5231,102164,574,66058,3766,24582,17109,573,33365,575,832,16759,235265,235248,108,235287,5231,10716,181430,235256,66058,7640,2611,919,476,25270,35571,1178,1156,9276,575,573,12918,18303,235265,10323,576,736,235269,665,6803,30390,1683,978,731,573,2681,24582,1178,5543,95178,1154,3118,689,10436,235265,109,688,2339,235269,1212,783,1443,66058,109,651,30390,3868,2611,26676,1167,4628,774,832,16759,235269,3547,573,8203,4824,3868,235265,2625,48825,578,22097,235269,573,33365,37845,1593,978,576,573,13795,235265,1417,3454,573,3868,2611,603,30390,3024,1693,978,235269,15267,573,5543,95178,591,165117,578,72638,235275,577,6378,1167,4628,235269,10241,1941,4964,9276,235265,110,5331,682,1230,1013,692,791,1089,1156,3920,235341],"total_duration":81320587916,"load_duration":42868270,"prompt_eval_count":15,"prompt_eval_duration":3107284000,"eval_count":210,"eval_duration":78169448000}
(gabriel) Gabrielui-MacBookPro:~ gabriel$
REST 엔드포인트를 통해 Gemma와 채팅
(gabriel) Gabrielui-MacBookPro:~ gabriel$ curl http://localhost:11434/api/chat -d '{
> "model": "gemma2",
> "messages": [
> {
> "role": "user",
> "content": "why is the sky blue?"
> }
> ]
> }'
{"model":"gemma2","created_at":"2024-09-05T15:32:34.118301Z","message":{"role":"assistant","content":"The"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:34.466319Z","message":{"role":"assistant","content":" sky"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:34.821353Z","message":{"role":"assistant","content":" appears"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:35.182394Z","message":{"role":"assistant","content":" blue"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:32:35.523289Z","message":{"role":"assistant","content":" due"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:33:50.644094Z","message":{"role":"assistant","content":" if"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:33:51.039595Z","message":{"role":"assistant","content":" you"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:33:51.411561Z","message":{"role":"assistant","content":" have"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:33:51.815174Z","message":{"role":"assistant","content":" any"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:33:52.152768Z","message":{"role":"assistant","content":" other"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:33:52.492949Z","message":{"role":"assistant","content":" questions"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:33:52.915517Z","message":{"role":"assistant","content":"!"},"done":false}
{"model":"gemma2","created_at":"2024-09-05T15:33:53.27064Z","message":{"role":"assistant","content":""},"done_reason":"stop","done":true,"total_duration":80946825634,"load_duration":44007406,"prompt_eval_count":15,"prompt_eval_duration":1749943000,"eval_count":190,"eval_duration":79152305000}
(gabriel) Gabrielui-MacBookPro:~ gabriel$
Ollama + Gemma 1.1 (2B)
(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama run gemma:2b
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling manifest
pulling c1864a5eb193... 100% ▕██████████████████████████████████████████████████████████████████████████████████▏ 1.7 GB
pulling 097a36493f71... 100% ▕██████████████████████████████████████████████████████████████████████████████████▏ 8.4 KB
pulling 109037bec39c... 100% ▕██████████████████████████████████████████████████████████████████████████████████▏ 136 B
pulling 22a838ceb7fb... 100% ▕██████████████████████████████████████████████████████████████████████████████████▏ 84 B
pulling 887433b89a90... 100% ▕██████████████████████████████████████████████████████████████████████████████████▏ 483 B
verifying sha256 digest
writing manifest
success
>>> Send a message (/? for help)
다운로드한 모델을 확인합니다.
(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama list
NAME ID SIZE MODIFIED
gemma:2b b50d6c999e59 1.7 GB 3 minutes ago
gemma2:latest ff02c3702f32 5.4 GB 40 minutes ago
(gabriel) Gabrielui-MacBookPro:~ gabriel$
Gemma2와 동일한 질문을 합니다. "why is the sky blue?"
>>> why is the sky blue?
The sky appears blue due to Rayleigh scattering.
**Rayleigh scattering** is a scattering phenomenon where light waves are scattered in different directions depending on their wavelength.
Blue light has a shorter wavelength than other colors of light, meaning it scatters more effectively. This is why the sky appears blue.
Here's a more detailed explanation:
* **Sunlight** contains all colors of the rainbow.
* **Blue light** has a shorter wavelength than other colors.
* **When blue light** interacts with molecules in the atmosphere, it **scatters more strongly** than other colors.
* This scattering causes blue light to be scattered **towards the observer**.
* The scattered blue light is what we see in the sky as **blue color**.
**Other factors that contribute to the blue color of the sky:**
* **Dust particles:** Air molecules can also scatter light, but their scattering effect is generally weaker than that of molecules in the
atmosphere.
* **Water droplets:** Water droplets in the atmosphere can also scatter light, but their effect is typically less significant than that of
molecules.
* **Sun's position:** The sky appears blue because the sun is lower in the sky, meaning it is directly overhead. This means that more blue
light reaches our eyes.
In summary, the blue color of the sky is due to Rayleigh scattering of light by molecules in the atmosphere, which primarily results from
the shorter wavelength of blue light.
>>> Send a message (/? for help)
실행시에 MacBook의 성능을 확인합니다.
실행한 모델 정보 확인
>>> /show info
Model
arch gemma
parameters 3B
quantization Q4_0
context length 8192
embedding length 2048
Parameters
repeat_penalty 1
stop "<start_of_turn>"
stop "<end_of_turn>"
License
Gemma Terms of Use
Last modified: February 21, 2024
>>> Send a message (/? for help)
Ollama + Gemma 1.1 (7B)
이번엔 Gemma 7B 모델을다운받아 테스트합니다.
(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama run gemma:7b
pulling manifest
pulling ef311de6af9d... 74% ▕█████████████████████████████████████████
pulling manifest
pulling ef311de6af9d... 74% ▕█████████████████████████████████████████
pulling manifest
pulling ef311de6af9d... 74% ▕█████████████████████████████████████████
pulling manifest
pulling ef311de6af9d... 100% ▕███████████████████████████████████████████████████████████████▏ 5.0 GB
pulling 097a36493f71... 100% ▕███████████████████████████████████████████████████████████████▏ 8.4 KB
pulling 109037bec39c... 100% ▕███████████████████████████████████████████████████████████████▏ 136 B
pulling 65bb16cf5983... 100% ▕███████████████████████████████████████████████████████████████▏ 109 B
pulling 0c2a5137eb3c... 100% ▕███████████████████████████████████████████████████████████████▏ 483 B
verifying sha256 digest
writing manifest
success
>>> Send a message (/? for help)
다운로드한 모델 확인
(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama list
NAME ID SIZE MODIFIED
gemma:7b a72c7f4d0a15 5.0 GB 2 weeks ago
gemma:2b b50d6c999e59 1.7 GB 2 weeks ago
gemma2:latest ff02c3702f32 5.4 GB 2 weeks ago
Gemma2와 동일한 질문을 합니다. "why is the sky blue?"
>>> why is the sky blue?
**The sky is blue due to a phenomenon called Rayleigh scattering.**
* Sunlight is composed of all the colors of the rainbow, each with a different wavelength.
* Shorter wavelengths of light, such as blue light, have higher energy and more tightly spaced photons.
* When sunlight interacts with molecules in the atmosphere, such as nitrogen and oxygen, the shorter wavelengths are
scattered more effectively.
**Here's how it works:**
1. Sunlight enters the atmosphere and interacts with molecules of nitrogen and oxygen.
2. These molecules have electrons that can be excited by the energy of the incoming light.
3. The electrons absorb the energy and vibrate, then emit it in all directions.
4. However, **blue light is scattered more efficiently than other colors** because its shorter wavelength allows for more
frequent collisions between the photons and the molecules.
**This scattering process results in:**
* More blue light is scattered in all directions, reaching our eyes and making the sky appear blue.
* Longer wavelengths of light, such as red light, are scattered less efficiently, so we see more of them directly from
the sun.
**Additional factors influencing the sky's color:**
* **Time of day:** The sky is generally bluer at midday when the sunlight travels through less atmosphere.
* **Cloud cover:** Clouds can block the sunlight and scatter less blue light, making the sky appear less blue.
* **Pollution:** Pollution in the atmosphere can also scatter light and affect the sky's color.
>>> Send a message (/? for help)
실행한 모델 정보 확인
>>> /show info
Model
arch gemma
parameters 9B
quantization Q4_0
context length 8192
embedding length 3072
Parameters
penalize_newline false
repeat_penalty 1
stop "<start_of_turn>"
stop "<end_of_turn>"
License
Gemma Terms of Use
Last modified: February 21, 2024
>>> Send a message (/? for help)
질의 성능 확인
Intel CPU의 MacBook에서 Gemma2 9B 모델 그리고 Gemma 2B, 7B 모델까지 Ollama로 구동해 봤습니다.
감사합니다.
'AI > LLM-Gemma' 카테고리의 다른 글
| Google Colab에서 Ollama를 사용하여 Gemma 모델을 실행 (0) | 2024.09.22 |
|---|---|
| Colab에서 T4 GPU 런타임설정 및 Kaggle Credential Key 설정 (0) | 2024.09.19 |