Here is the code link:
https://github.com/FassyGit/LightFM_liu/blob/master/U_F1.py
I use NYC data as other experimens.
The split of the training data was seperated by the timeline, and I have normalised the interaction matrix by replacing the checkin frequencies with the checkin frequencies percentage which range between 0 and 1.
And I use this mormalized matrix and lighfm model to train the data, what I got was a little worse than the original data,here is the out come :
The model was trained with warp loss function
I am beginning to model model has been fitted this is the model that consider the checkin times Time used: 4.910935999999992 Train_auc is 0.999486 Test_auc is 0.763801 train_pm_auc is 0.860700, test_pm_auc is 0.685053 /home/s2013258/.local/lib/python3.5/site-packages/sklearn/cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20. "This module will be removed in 0.20.", DeprecationWarning)
In theory, warp loss fuction take the input data as binary, there should not be any difference, but the actual result was a little worse...
I think it should have something to do with the loss that this model use in updates.
Then I use the normalised matrix to achieve collabrative filtering, and up to now, the evaluation metric I use is rmse.
But considering the recommendation object, I will use pre@k instead.
But I have not done that yet...
I use pre@k as metrics, and the results were not so good, this is due to the split of the data, I cut the whole data with the same time. As a result, there is some users who got no check in history in the train data, and some users got little checkins in test. which results in the low pre@k
Here is the results...
the friend is 1 and the corresponding prek is 0.172853 the friend is 2 and the corresponding prek is 0.202955 the friend is 3 and the corresponding prek is 0.213019 the friend is 4 and the corresponding prek is 0.217544 the friend is 5 and the corresponding prek is 0.219021 the friend is 6 and the corresponding prek is 0.221145 the friend is 7 and the corresponding prek is 0.222161 the friend is 8 and the corresponding prek is 0.222622 the friend is 9 and the corresponding prek is 0.222161 the friend is 10 and the corresponding prek is 0.223084 the friend is 11 and the corresponding prek is 0.223730 the friend is 12 and the corresponding prek is 0.223546 the friend is 13 and the corresponding prek is 0.223084 the friend is 14 and the corresponding prek is 0.222992 the friend is 15 and the corresponding prek is 0.223269 the friend is 16 and the corresponding prek is 0.223823 the friend is 17 and the corresponding prek is 0.223453 the friend is 18 and the corresponding prek is 0.223269 the friend is 19 and the corresponding prek is 0.223638 the friend is 20 and the corresponding prek is 0.223638 the friend is 21 and the corresponding prek is 0.223546 the friend is 22 and the corresponding prek is 0.223176 the friend is 23 and the corresponding prek is 0.224100 the friend is 24 and the corresponding prek is 0.223269 the friend is 25 and the corresponding prek is 0.223453 the friend is 26 and the corresponding prek is 0.223546 the friend is 27 and the corresponding prek is 0.223823 the friend is 28 and the corresponding prek is 0.223269 the friend is 29 and the corresponding prek is 0.223453 the friend is 30 and the corresponding prek is 0.223453 the friend is 31 and the corresponding prek is 0.223269 the friend is 32 and the corresponding prek is 0.223084 the friend is 33 and the corresponding prek is 0.222899 the friend is 34 and the corresponding prek is 0.223546 the friend is 35 and the corresponding prek is 0.223361 the friend is 36 and the corresponding prek is 0.223269 the friend is 37 and the corresponding prek is 0.223176 the friend is 38 and the corresponding prek is 0.223269 the friend is 39 and the corresponding prek is 0.224007 the friend is 40 and the corresponding prek is 0.223638 the friend is 41 and the corresponding prek is 0.223269 the friend is 42 and the corresponding prek is 0.223269 the friend is 43 and the corresponding prek is 0.223176 the friend is 44 and the corresponding prek is 0.224377 the friend is 45 and the corresponding prek is 0.223638 the friend is 46 and the corresponding prek is 0.223176 the friend is 47 and the corresponding prek is 0.223361 the friend is 48 and the corresponding prek is 0.223453 the friend is 49 and the corresponding prek is 0.223638 the friend is 50 and the corresponding prek is 0.223546 the friend is 51 and the corresponding prek is 0.223546 the friend is 52 and the corresponding prek is 0.223730 the friend is 53 and the corresponding prek is 0.223638 the friend is 54 and the corresponding prek is 0.223269 the friend is 55 and the corresponding prek is 0.223269 the friend is 56 and the corresponding prek is 0.223546 the friend is 57 and the corresponding prek is 0.223084 the friend is 58 and the corresponding prek is 0.223453 the friend is 59 and the corresponding prek is 0.223730 the friend is 60 and the corresponding prek is 0.223915 the friend is 61 and the corresponding prek is 0.223269 the friend is 62 and the corresponding prek is 0.223823 the friend is 63 and the corresponding prek is 0.223084 the friend is 64 and the corresponding prek is 0.223546 the friend is 65 and the corresponding prek is 0.223453 the friend is 66 and the corresponding prek is 0.223730 the friend is 67 and the corresponding prek is 0.223361 the friend is 68 and the corresponding prek is 0.223546 the friend is 69 and the corresponding prek is 0.223730 the friend is 70 and the corresponding prek is 0.223730 the friend is 71 and the corresponding prek is 0.223453 the friend is 72 and the corresponding prek is 0.223269 the friend is 73 and the corresponding prek is 0.223638 the friend is 74 and the corresponding prek is 0.223269 the friend is 75 and the corresponding prek is 0.223453 the friend is 76 and the corresponding prek is 0.223546 the friend is 77 and the corresponding prek is 0.223638 the friend is 78 and the corresponding prek is 0.223361 the friend is 79 and the corresponding prek is 0.223730 the friend is 80 and the corresponding prek is 0.223638 the friend is 81 and the corresponding prek is 0.223084 the friend is 82 and the corresponding prek is 0.223453 the friend is 83 and the corresponding prek is 0.222992 the friend is 84 and the corresponding prek is 0.223638 the friend is 85 and the corresponding prek is 0.222899 the friend is 86 and the corresponding prek is 0.223269 the friend is 87 and the corresponding prek is 0.223176 the friend is 88 and the corresponding prek is 0.223823 the friend is 89 and the corresponding prek is 0.223084 the friend is 90 and the corresponding prek is 0.223823 the friend is 91 and the corresponding prek is 0.223638 the friend is 92 and the corresponding prek is 0.223176 the friend is 93 and the corresponding prek is 0.222807 the friend is 94 and the corresponding prek is 0.223546 the friend is 95 and the corresponding prek is 0.223453 the friend is 96 and the corresponding prek is 0.223823 the friend is 97 and the corresponding prek is 0.223823 the friend is 98 and the corresponding prek is 0.223084 the friend is 99 and the corresponding prek is 0.223546 the usr percentage is 0.000000 and the corresponding prek is 0.020129 the usr percentage is 0.100000 and the corresponding prek is 0.197045 the usr percentage is 0.200000 and the corresponding prek is 0.214035 the usr percentage is 0.300000 and the corresponding prek is 0.219298 the usr percentage is 0.400000 and the corresponding prek is 0.221607 the usr percentage is 0.500000 and the corresponding prek is 0.223084 the usr percentage is 0.600000 and the corresponding prek is 0.223269 the usr percentage is 0.700000 and the corresponding prek is 0.223638 the usr percentage is 0.800000 and the corresponding prek is 0.223730 the usr percentage is 0.900000 and the corresponding prek is 0.223638 the k is 1 and the corresponding prek is 0.462604 the k is 2 and the corresponding prek is 0.405817 the k is 3 and the corresponding prek is 0.364728 the k is 4 and the corresponding prek is 0.325946 the k is 5 and the corresponding prek is 0.300462 the k is 6 and the corresponding prek is 0.279624 the k is 7 and the corresponding prek is 0.261047 the k is 8 and the corresponding prek is 0.246191 the k is 9 and the corresponding prek is 0.234534 the k is 10 and the corresponding prek is 0.223084 the k is 11 and the corresponding prek is 0.211702 the k is 12 and the corresponding prek is 0.201447 the k is 13 and the corresponding prek is 0.193338 the k is 14 and the corresponding prek is 0.185134 the k is 15 and the corresponding prek is 0.177839 the k is 16 and the corresponding prek is 0.170533 the k is 17 and the corresponding prek is 0.163869 the k is 18 and the corresponding prek is 0.157741 the k is 19 and the corresponding prek is 0.152209 the k is 20 and the corresponding prek is 0.146814 the k is 21 and the corresponding prek is 0.142373 the k is 22 and the corresponding prek is 0.137791 the k is 23 and the corresponding prek is 0.134088 the k is 24 and the corresponding prek is 0.130540 the k is 25 and the corresponding prek is 0.126759 the k is 26 and the corresponding prek is 0.123411 the k is 27 and the corresponding prek is 0.120311 the k is 28 and the corresponding prek is 0.117663 the k is 29 and the corresponding prek is 0.114847 the k is 30 and the corresponding prek is 0.111942 the k is 31 and the corresponding prek is 0.109463 the k is 32 and the corresponding prek is 0.107341 the k is 33 and the corresponding prek is 0.105123 the k is 34 and the corresponding prek is 0.102873 the k is 35 and the corresponding prek is 0.100725 the k is 36 and the corresponding prek is 0.098723 the k is 37 and the corresponding prek is 0.096703 the k is 38 and the corresponding prek is 0.095033 the k is 39 and the corresponding prek is 0.093449 the k is 40 and the corresponding prek is 0.091644 the k is 41 and the corresponding prek is 0.090016 the k is 42 and the corresponding prek is 0.088445 the k is 43 and the corresponding prek is 0.087140 the k is 44 and the corresponding prek is 0.085768 the k is 45 and the corresponding prek is 0.084416 the k is 46 and the corresponding prek is 0.083042 the k is 47 and the corresponding prek is 0.081826 the k is 48 and the corresponding prek is 0.080602 the k is 49 and the corresponding prek is 0.079390
1. the friend reoresents the friend number we choose for a user
2. the user percentage represent the percentage user takes
3. the k is the k in pre@k
We can see that the pre@k is actually very low because of the cold start problem
To sovle this problem, I choose a different way to seprating the data, that is to choose evey user's 80% checkin history as train and the remaining 20% as test, then there will be no such problem
This really works well:
Here is the results:
the friend is 1 and the corresponding prek is 0.328440 the friend is 2 and the corresponding prek is 0.387719 the friend is 3 and the corresponding prek is 0.409511 the friend is 4 and the corresponding prek is 0.414589 the friend is 5 and the corresponding prek is 0.420314 the friend is 6 and the corresponding prek is 0.424007 the friend is 7 and the corresponding prek is 0.427978 the friend is 8 and the corresponding prek is 0.427886 the friend is 9 and the corresponding prek is 0.429732 the friend is 10 and the corresponding prek is 0.430379 the friend is 11 and the corresponding prek is 0.429271 the friend is 12 and the corresponding prek is 0.431210 the friend is 13 and the corresponding prek is 0.430840 the friend is 14 and the corresponding prek is 0.430933 the friend is 15 and the corresponding prek is 0.430286 the friend is 16 and the corresponding prek is 0.431117 the friend is 17 and the corresponding prek is 0.430656 the friend is 18 and the corresponding prek is 0.431856 the friend is 19 and the corresponding prek is 0.431302 the friend is 20 and the corresponding prek is 0.431302 the friend is 21 and the corresponding prek is 0.431117 the friend is 22 and the corresponding prek is 0.432502 the friend is 23 and the corresponding prek is 0.432133 /home/s2013258/.local/lib/python3.5/site-packages/sklearn/cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20. "This module will be removed in 0.20.", DeprecationWarning) the friend is 24 and the corresponding prek is 0.430933 the friend is 25 and the corresponding prek is 0.430563 the friend is 26 and the corresponding prek is 0.430379 the friend is 27 and the corresponding prek is 0.430840 the friend is 28 and the corresponding prek is 0.431764 the friend is 29 and the corresponding prek is 0.431302 the friend is 30 and the corresponding prek is 0.431671 the friend is 31 and the corresponding prek is 0.431210 the friend is 32 and the corresponding prek is 0.430563 the friend is 33 and the corresponding prek is 0.432041 the friend is 34 and the corresponding prek is 0.431487 the friend is 35 and the corresponding prek is 0.430656 the friend is 36 and the corresponding prek is 0.431671 the friend is 37 and the corresponding prek is 0.431117 the friend is 38 and the corresponding prek is 0.431579 the friend is 39 and the corresponding prek is 0.431394 the friend is 40 and the corresponding prek is 0.431764 the friend is 41 and the corresponding prek is 0.431302 the friend is 42 and the corresponding prek is 0.430933 the friend is 43 and the corresponding prek is 0.430286 the friend is 44 and the corresponding prek is 0.431302 the friend is 45 and the corresponding prek is 0.430933 the friend is 46 and the corresponding prek is 0.431117 the friend is 47 and the corresponding prek is 0.431487 the friend is 48 and the corresponding prek is 0.431302 the friend is 49 and the corresponding prek is 0.430656 the friend is 50 and the corresponding prek is 0.431487 the friend is 51 and the corresponding prek is 0.431764 the friend is 52 and the corresponding prek is 0.430102 the friend is 53 and the corresponding prek is 0.431579 the friend is 54 and the corresponding prek is 0.431210 the friend is 55 and the corresponding prek is 0.431856 the friend is 56 and the corresponding prek is 0.431487 the friend is 57 and the corresponding prek is 0.430656 the friend is 58 and the corresponding prek is 0.431210 the friend is 59 and the corresponding prek is 0.430840 the friend is 60 and the corresponding prek is 0.431394 the friend is 61 and the corresponding prek is 0.431210 the friend is 62 and the corresponding prek is 0.430840 the friend is 63 and the corresponding prek is 0.431579 the friend is 64 and the corresponding prek is 0.430933 the friend is 65 and the corresponding prek is 0.430563 the friend is 66 and the corresponding prek is 0.430840 the friend is 67 and the corresponding prek is 0.431487 the friend is 68 and the corresponding prek is 0.430840 the friend is 69 and the corresponding prek is 0.431302 the friend is 70 and the corresponding prek is 0.431764 the friend is 71 and the corresponding prek is 0.431394 the friend is 72 and the corresponding prek is 0.431487 the friend is 73 and the corresponding prek is 0.431117 the friend is 74 and the corresponding prek is 0.431671 the friend is 75 and the corresponding prek is 0.430933 the friend is 76 and the corresponding prek is 0.431210 the friend is 77 and the corresponding prek is 0.431302 the friend is 78 and the corresponding prek is 0.431117 the friend is 79 and the corresponding prek is 0.431394 the friend is 80 and the corresponding prek is 0.431487 the friend is 81 and the corresponding prek is 0.430933 the friend is 82 and the corresponding prek is 0.431302 the friend is 83 and the corresponding prek is 0.431394 the friend is 84 and the corresponding prek is 0.430656 the friend is 85 and the corresponding prek is 0.430563 the friend is 86 and the corresponding prek is 0.431487 the friend is 87 and the corresponding prek is 0.430379 the friend is 88 and the corresponding prek is 0.430933 the friend is 89 and the corresponding prek is 0.431025 the friend is 90 and the corresponding prek is 0.431394 the friend is 91 and the corresponding prek is 0.430748 the friend is 92 and the corresponding prek is 0.430840 the friend is 93 and the corresponding prek is 0.430840 the friend is 94 and the corresponding prek is 0.430933 the friend is 95 and the corresponding prek is 0.431394 the friend is 96 and the corresponding prek is 0.431210 the friend is 97 and the corresponding prek is 0.431025 the friend is 98 and the corresponding prek is 0.430471 the friend is 99 and the corresponding prek is 0.430471 the usr percentage is 0.000000 and the corresponding prek is 0.037027 the usr percentage is 0.100000 and the corresponding prek is 0.377747 the usr percentage is 0.200000 and the corresponding prek is 0.414127 the usr percentage is 0.300000 and the corresponding prek is 0.422992 the usr percentage is 0.400000 and the corresponding prek is 0.427886 the usr percentage is 0.500000 and the corresponding prek is 0.430379 the usr percentage is 0.600000 and the corresponding prek is 0.431394 the usr percentage is 0.700000 and the corresponding prek is 0.431487 the usr percentage is 0.800000 and the corresponding prek is 0.431856 the usr percentage is 0.900000 and the corresponding prek is 0.431948 the k is 2 and the corresponding prek is 0.714220 the k is 3 and the corresponding prek is 0.656202 the k is 4 and the corresponding prek is 0.605494 the k is 5 and the corresponding prek is 0.566020 the k is 6 and the corresponding prek is 0.531086 the k is 7 and the corresponding prek is 0.502704 the k is 8 and the corresponding prek is 0.475531 the k is 9 and the corresponding prek is 0.453575 the k is 10 and the corresponding prek is 0.430379 the k is 11 and the corresponding prek is 0.410896 the k is 12 and the corresponding prek is 0.394506 the k is 13 and the corresponding prek is 0.377939 the k is 14 and the corresponding prek is 0.362353 the k is 15 and the corresponding prek is 0.348169 the k is 16 and the corresponding prek is 0.334661 the k is 17 and the corresponding prek is 0.322633 the k is 18 and the corresponding prek is 0.311429 the k is 19 and the corresponding prek is 0.301453 the k is 20 and the corresponding prek is 0.291690 the k is 21 and the corresponding prek is 0.282856 the k is 22 and the corresponding prek is 0.274070 the k is 23 and the corresponding prek is 0.266731 the k is 24 and the corresponding prek is 0.260042 the k is 25 and the corresponding prek is 0.253186 the k is 26 and the corresponding prek is 0.246928 the k is 27 and the corresponding prek is 0.240860 the k is 28 and the corresponding prek is 0.235226 the k is 29 and the corresponding prek is 0.229630 the k is 30 and the corresponding prek is 0.223730 the k is 31 and the corresponding prek is 0.218658 the k is 32 and the corresponding prek is 0.214104 the k is 33 and the corresponding prek is 0.209267 the k is 34 and the corresponding prek is 0.204905 the k is 35 and the corresponding prek is 0.201003 the k is 36 and the corresponding prek is 0.197497 the k is 37 and the corresponding prek is 0.193482 the k is 38 and the corresponding prek is 0.190139 the k is 39 and the corresponding prek is 0.186614 the k is 40 and the corresponding prek is 0.183518 the k is 41 and the corresponding prek is 0.180393 the k is 42 and the corresponding prek is 0.177549 the k is 43 and the corresponding prek is 0.174601 the k is 44 and the corresponding prek is 0.171997 the k is 45 and the corresponding prek is 0.169303 the k is 46 and the corresponding prek is 0.166486 the k is 47 and the corresponding prek is 0.164064 the k is 48 and the corresponding prek is 0.161723 the k is 49 and the corresponding prek is 0.159233
From this result, we can see that the friend number more than 10 and the user percentage more than 60% to 70% does not makes much difference.
TBC