Repository navigation
Op type not registered 'Addons>ParseTime' #274
Description
Issue
Pipeline stored to disk does not construct the entire transformation graph unless Lambda layer is manually invoked.
Steps to reproduce:
Clone the repo
git clone https://github.com/pritamdodeja/tft_tasks
cd tft_tasks
git checkout 732b0
Run the entire lifecycle of supported tasks to create the various artifacts
python tft_tasks.py --task clean_directory --task write_raw_tfrecords --task view_original_sample_data --task transform_tfrecords --task view_transformed_sample_data --task train_non_embedding_model --task train_embedding_model
Comment out lines 434 to 437 in tft_tasks.py containing the following:
434 layers.Lambda(
435 fn_seconds_since_1970,
436 name="seconds_since_1970")(
437 transformed_inputs['pickup_datetime'])
Run the following:
python tft_tasks.py --task view_transformed_sample_data
Get the following error:
FileNotFoundError: Op type not registered 'Addons>ParseTime' in binary running on fedora. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) `tf.contrib.resampler` should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.
You may be trying to load on a different device from the computational device. Consider setting the `experimental_io_device` option in `tf.saved_model.LoadOptions` to the io_device such as '/job:localhost'.
If you un-comment those lines and re-run the same command again, it runs fine. My understanding of tft was that if you write preprocessing_fn following the various rules, that it will capture the entire set of transformations for you. Why isn't the Lambda layer captured, and what is the right thing to do in a situation such as this?
Thank you!
Activity
pritamdodeja commentedon May 25, 2022
@pindinagesh I was able to solve this issue. This has to do with serialization issues related to Lambda layers. I do not know if that is a feature request somewhere, and if so, where. I do not technically understand why Lambda layers cannot be serialized and who would work on that problem. This information was missing in the getting started notebook. I have submitted pr275 to update that notebook. It may be worthwhile making that notebook more prominent in the repo as it is the best place to get started. Let me know if you need a corresponding issue for pr275 or if this issue suffices.
Thank you!
pindinagesh commentedon May 26, 2022
Hi @pindinagesh
Glad you are able to solve the issue, can you please move this issue to closed status since this issue is resolved. Thanks!
pritamdodeja commentedon May 26, 2022
Hi @pindinagesh - happy to close this, but how did you want to handle pr275?
pindinagesh commentedon May 28, 2022
Hi @pritamdodeja
Thanks for the confirmation, I have assigned this PR to the reviewer, once it's approved it will be merged