Skip to content

Mismatch in Hyper Parameters provided in code and those specified in the paper #44

Open
@ghost

Description

Not really an issue, more of a question.

I am carrying out some research on sentiment analysis and I was looking to reproduce the results of Subtask A, specified in the paper here. Now, I should add that with the source code as is provided, and with TensorFlow backend, I get very similar results. However, upon closer inspection, I noticed some inconsistincies between what's in the code and in the paper.

For example, the paper states that:

We train all of our networks to minimize the
cross-entropy loss, using back-propagation with
stochastic gradient descent and mini-batches of
size 128.

However, in the code, the batch size is 50:

history = nn_model.fit(training[0], training[1], validation_data=validation if not FINAL else testing, epochs=50, batch_size=50, class_weight=class_weights, callbacks=_callbacks)

There are few other inconsistencies as well. My question is, should I change the hyper parameters to the ones specified in the paper in order to exactly replicate the results? Or is exact replication not possible? I should also add that when I run the system multiple times, I get similar, but slightly different results for each epoch. Maybe this has something to with using TensorFlow as a backend? I've been reading that some people have had issues getting the same results when using TensorFlow backend.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions