Fix pickle issues #845
Merged
Fix pickle issues #845
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
X needs to be a dataframe currently. Print the type of X in the exception message so it's clearer to users what is not supported.
_hazard() was a dynamic lambda function - make it a normal function so it can be pickled.
For _create_neg_likelihood_with_penalty_function() use the functools.partial approach instead of function inside function so that it can be pickled correctly. Also, save the _initial_point_dict so that the unflatten function can be generated on the fly and the autograd function does not need to be a property that needs to be saved.
|
This is A+++++ 🙌 I'm going to point this PR at my release branch and get it out soon. I'd like to do some local testing with it too. |
|
Sounds great, thanks! |
|
Hi @AbdealiJK !!! good post. I have not been able to save/serialize my WeibullAFTFitter. It keeps saying "" Sorry, pickling not yet supported. See pydata/patsy#26 if you want to help." I have been able to save it with a CoxPHFitter, but not with WeibullAFTFitter. And this is the one wich shows best results. |
|
Commented in #1138 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For univariate models:
_hazardis a property in the object, but actually a function - By making it a function, it becomes a part of the class definition and hence the pickle does not consider it as a property that needs to be serialized.From my manual testing, these fitters can now be pickled:
Also, removed the xfail from the pickle and added joblib test cases in Univariate.
EDIT: I did some more digging and figured out the issue for the Regression models too.
For regression models:
_neg_likelihood_with_penalty_functionis a property in the object, but actually a function. By making it a functools.partial, pickle will know how to pickle it. Also, it needs theunflattenfunction which is an autograd function which cannot be pickled. So, we save the data to recreate it with autograd instead of saving the autograd function itself.From my manual testing, these fitters can now be pickled:
And I also added pickle test cases for this, and removed the xfail from joblib test case for Regression models
Fixes #188