sample_weight is negative when running kmeans clustering #61
Description
This is a really nice work!
I followed the instruction to quantize Llama-2-7b-chat-hf. At kmeans clustering step, I ran the following command:
python nuq.py --bit 4 --model_type llama --model ~/models/${model_name}-squeezellm/model_chunks --gradient ~/models/${model_name}-squeezellm/gradient_chunks --output ~/models/${model_name}-squeezellm/LUT
And got this error:
Quantizing layers [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
Quantizing layer 0
0%| | 0/7 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/home/mmilin/projects/ming_benchmark_vllm/SqueezeLLM/quantization/nuq.py", line 166, in <module>
kmeans = KMeans(
File "/home/mmilin/projects/ming_benchmark_vllm/venv/lib/python3.9/site-packages/sklearn/base.py", line 1152, in wrapper
return fit_method(estimator, *args, **kwargs)
File "/home/mmilin/projects/ming_benchmark_vllm/venv/lib/python3.9/site-packages/sklearn/cluster/_kmeans.py", line 1519, in fit
centers_init = self._init_centroids(
File "/home/mmilin/projects/ming_benchmark_vllm/venv/lib/python3.9/site-packages/sklearn/cluster/_kmeans.py", line 1019, in _init_centroids
centers, _ = _kmeans_plusplus(
File "/home/mmilin/projects/ming_benchmark_vllm/venv/lib/python3.9/site-packages/sklearn/cluster/_kmeans.py", line 229, in _kmeans_plusplus
center_id = random_state.choice(n_samples, p=sample_weight / sample_weight.sum())
File "numpy/random/mtrand.pyx", line 973, in numpy.random.mtrand.RandomState.choice
ValueError: probabilities are not non-negative
I manually checked the sample_weight. It has negative elements, which is wired.