Skip to content

sample_weight is negative when running kmeans clustering #61

@MingLin-home

Description

@MingLin-home

This is a really nice work!

I followed the instruction to quantize Llama-2-7b-chat-hf. At kmeans clustering step, I ran the following command:

python nuq.py --bit 4 --model_type llama --model ~/models/${model_name}-squeezellm/model_chunks --gradient ~/models/${model_name}-squeezellm/gradient_chunks --output ~/models/${model_name}-squeezellm/LUT

And got this error:

Quantizing layers [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
Quantizing layer 0
  0%|                                                                                                  | 0/7 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/mmilin/projects/ming_benchmark_vllm/SqueezeLLM/quantization/nuq.py", line 166, in <module>
    kmeans = KMeans(
  File "/home/mmilin/projects/ming_benchmark_vllm/venv/lib/python3.9/site-packages/sklearn/base.py", line 1152, in wrapper
    return fit_method(estimator, *args, **kwargs)
  File "/home/mmilin/projects/ming_benchmark_vllm/venv/lib/python3.9/site-packages/sklearn/cluster/_kmeans.py", line 1519, in fit
    centers_init = self._init_centroids(
  File "/home/mmilin/projects/ming_benchmark_vllm/venv/lib/python3.9/site-packages/sklearn/cluster/_kmeans.py", line 1019, in _init_centroids
    centers, _ = _kmeans_plusplus(
  File "/home/mmilin/projects/ming_benchmark_vllm/venv/lib/python3.9/site-packages/sklearn/cluster/_kmeans.py", line 229, in _kmeans_plusplus
    center_id = random_state.choice(n_samples, p=sample_weight / sample_weight.sum())
  File "numpy/random/mtrand.pyx", line 973, in numpy.random.mtrand.RandomState.choice
ValueError: probabilities are not non-negative

I manually checked the sample_weight. It has negative elements, which is wired.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions