text:
Expose tfa.types doc

Description

Fixes #2132. The exposed namespace aligns to the one of core TF: tf.types.experiemental.* vs tfa.types.*.
Use explicit_package_contents_filter to filter out meaningless endpoints. Like the module down below will not be generated.
Pin to newest tensorflow_docs version. This is discussion needed. IMO, it doesn't really matter on our end unless docs team has some breaking change on the interface the API generator.

Diff TOC

diff --git a/home/windqaq/addons_api/_toc.yaml b/home/windqaq/new_addons_api/_toc.yaml
index 7c9c58a..45ee1f7 100644
--- a/home/windqaq/addons_api/_toc.yaml
+++ b/home/windqaq/new_addons_api/_toc.yaml
@@ -3,6 +3,8 @@ toc:
   section:
   - title: Overview
     path: /addons/api_docs/python/tfa
+  - title: register_all
+    path: /addons/api_docs/python/tfa/register_all
 - title: tfa.activations
   section:
   - title: Overview
@@ -35,18 +37,6 @@ toc:
     path: /addons/api_docs/python/tfa/callbacks/TimeStopping
   - title: TQDMProgressBar
     path: /addons/api_docs/python/tfa/callbacks/TQDMProgressBar
-  - title: average_model_checkpoint
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/callbacks/average_model_checkpoint
-  - title: time_stopping
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/callbacks/time_stopping
-  - title: tqdm_progress_bar
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/callbacks/tqdm_progress_bar
 - title: tfa.image
   section:
   - title: Overview
@@ -97,76 +87,6 @@ toc:
     path: /addons/api_docs/python/tfa/image/translate
   - title: translate_xy
     path: /addons/api_docs/python/tfa/image/translate_xy
-  - title: color_ops
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/color_ops
-    - title: equalize_image
-      path: /addons/api_docs/python/tfa/image/color_ops/equalize_image
-    - title: Number
-      path: /addons/api_docs/python/tfa/image/color_ops/Number
-    - title: sharpness_image
-      path: /addons/api_docs/python/tfa/image/color_ops/sharpness_image
-    - title: TensorLike
-      path: /addons/api_docs/python/tfa/image/color_ops/TensorLike
-  - title: compose_ops
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/compose_ops
-  - title: cutout_ops
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/cutout_ops
-  - title: distance_transform
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/distance_transform
-  - title: distort_image_ops
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/distort_image_ops
-  - title: filters
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/filters
-    - title: FloatTensorLike
-      path: /addons/api_docs/python/tfa/image/filters/FloatTensorLike
-  - title: resampler_ops
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/resampler_ops
-  - title: transform_ops
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/transform_ops
-    - title: angles_to_projective_transforms
-      path: /addons/api_docs/python/tfa/image/transform_ops/angles_to_projective_transforms
-    - title: compose_transforms
-      path: /addons/api_docs/python/tfa/image/transform_ops/compose_transforms
-    - title: flat_transforms_to_matrices
-      path: /addons/api_docs/python/tfa/image/transform_ops/flat_transforms_to_matrices
-    - title: matrices_to_flat_transforms
-      path: /addons/api_docs/python/tfa/image/transform_ops/matrices_to_flat_transforms
-  - title: translate_ops
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/translate_ops
-    - title: translations_to_projective_transforms
-      path: /addons/api_docs/python/tfa/image/translate_ops/translations_to_projective_transforms
-  - title: utils
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/utils
-    - title: from_4D_image
-      path: /addons/api_docs/python/tfa/image/utils/from_4D_image
-    - title: get_ndims
-      path: /addons/api_docs/python/tfa/image/utils/get_ndims
-    - title: to_4D_image
-      path: /addons/api_docs/python/tfa/image/utils/to_4D_image
-    - title: unwrap
-      path: /addons/api_docs/python/tfa/image/utils/unwrap
-    - title: wrap
-      path: /addons/api_docs/python/tfa/image/utils/wrap
 - title: tfa.layers
   section:
   - title: Overview
@@ -215,76 +135,6 @@ toc:
     path: /addons/api_docs/python/tfa/layers/TLU
   - title: WeightNormalization
     path: /addons/api_docs/python/tfa/layers/WeightNormalization
-  - title: adaptive_pooling
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/adaptive_pooling
-    - title: AdaptivePooling1D
-      path: /addons/api_docs/python/tfa/layers/adaptive_pooling/AdaptivePooling1D
-    - title: AdaptivePooling2D
-      path: /addons/api_docs/python/tfa/layers/adaptive_pooling/AdaptivePooling2D
-    - title: AdaptivePooling3D
-      path: /addons/api_docs/python/tfa/layers/adaptive_pooling/AdaptivePooling3D
-  - title: esn
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/esn
-    - title: Activation
-      path: /addons/api_docs/python/tfa/layers/esn/Activation
-    - title: Initializer
-      path: /addons/api_docs/python/tfa/layers/esn/Initializer
-  - title: gelu
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/gelu
-  - title: maxout
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/maxout
-  - title: multihead_attention
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/multihead_attention
-  - title: normalizations
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/normalizations
-  - title: optical_flow
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/optical_flow
-  - title: poincare
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/poincare
-  - title: polynomial
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/polynomial
-  - title: snake
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/snake
-  - title: sparsemax
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/sparsemax
-  - title: spatial_pyramid_pooling
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/spatial_pyramid_pooling
-  - title: spectral_normalization
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/spectral_normalization
-  - title: tlu
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/tlu
-  - title: wrappers
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/wrappers
 - title: tfa.losses
   section:
   - title: Overview
@@ -331,42 +181,6 @@ toc:
     path: /addons/api_docs/python/tfa/losses/triplet_semihard_loss
   - title: WeightedKappaLoss
     path: /addons/api_docs/python/tfa/losses/WeightedKappaLoss
-  - title: contrastive
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/losses/contrastive
-  - title: focal_loss
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/losses/focal_loss
-  - title: kappa_loss
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/losses/kappa_loss
-  - title: lifted
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/losses/lifted
-  - title: metric_learning
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/losses/metric_learning
-    - title: angular_distance
-      path: /addons/api_docs/python/tfa/losses/metric_learning/angular_distance
-    - title: pairwise_distance
-      path: /addons/api_docs/python/tfa/losses/metric_learning/pairwise_distance
-  - title: npairs
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/losses/npairs
-  - title: quantiles
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/losses/quantiles
-  - title: triplet
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/losses/triplet
 - title: tfa.metrics
   section:
   - title: Overview
@@ -391,44 +205,6 @@ toc:
     path: /addons/api_docs/python/tfa/metrics/MultiLabelConfusionMatrix
   - title: RSquare
     path: /addons/api_docs/python/tfa/metrics/RSquare
-  - title: cohens_kappa
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/metrics/cohens_kappa
-    - title: AcceptableDTypes
-      path: /addons/api_docs/python/tfa/metrics/cohens_kappa/AcceptableDTypes
-  - title: f_scores
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/metrics/f_scores
-  - title: geometric_mean
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/metrics/geometric_mean
-  - title: hamming
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/metrics/hamming
-    - title: hamming_loss_fn
-      path: /addons/api_docs/python/tfa/metrics/hamming/hamming_loss_fn
-  - title: matthews_correlation_coefficient
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/metrics/matthews_correlation_coefficient
-  - title: multilabel_confusion_matrix
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/metrics/multilabel_confusion_matrix
-  - title: r_square
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/metrics/r_square
-  - title: utils
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/metrics/utils
-    - title: sample_weight_shape_match
-      path: /addons/api_docs/python/tfa/metrics/utils/sample_weight_shape_match
 - title: tfa.optimizers
   section:
   - title: Overview
@@ -469,76 +245,6 @@ toc:
     path: /addons/api_docs/python/tfa/optimizers/TriangularCyclicalLearningRate
   - title: Yogi
     path: /addons/api_docs/python/tfa/optimizers/Yogi
-  - title: average_wrapper
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/average_wrapper
-  - title: conditional_gradient
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/conditional_gradient
-  - title: cyclical_learning_rate
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/cyclical_learning_rate
-  - title: lamb
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/lamb
-  - title: lazy_adam
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/lazy_adam
-  - title: lookahead
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/lookahead
-  - title: moving_average
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/moving_average
-  - title: novograd
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/novograd
-  - title: proximal_adagrad
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/proximal_adagrad
-  - title: rectified_adam
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/rectified_adam
-  - title: stochastic_weight_averaging
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/stochastic_weight_averaging
-  - title: weight_decay_optimizers
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/weight_decay_optimizers
-    - title: DecoupledWeightDecayExtension
-      path: /addons/api_docs/python/tfa/optimizers/weight_decay_optimizers/DecoupledWeightDecayExtension
-  - title: yogi
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/yogi
-- title: tfa.options
-  section:
-  - title: Overview
-    path: /addons/api_docs/python/tfa/options
-  - title: warn_fallback
-    path: /addons/api_docs/python/tfa/options/warn_fallback
-- title: tfa.register
-  section:
-  - title: Overview
-    path: /addons/api_docs/python/tfa/register
-  - title: register_all
-    path: /addons/api_docs/python/tfa/register/register_all
-  - title: register_custom_kernels
-    path: /addons/api_docs/python/tfa/register/register_custom_kernels
-  - title: register_keras_objects
-    path: /addons/api_docs/python/tfa/register/register_keras_objects
 - title: tfa.rnn
   section:
   - title: Overview
@@ -553,26 +259,6 @@ toc:
     path: /addons/api_docs/python/tfa/rnn/NASCell
   - title: PeepholeLSTMCell
     path: /addons/api_docs/python/tfa/rnn/PeepholeLSTMCell
-  - title: esn_cell
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/rnn/esn_cell
-  - title: layer_norm_lstm_cell
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/rnn/layer_norm_lstm_cell
-  - title: layer_norm_simple_rnn_cell
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/rnn/layer_norm_simple_rnn_cell
-  - title: nas_cell
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/rnn/nas_cell
-  - title: peephole_lstm_cell
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/rnn/peephole_lstm_cell
 - title: tfa.seq2seq
   section:
   - title: Overview
@@ -641,40 +327,6 @@ toc:
     path: /addons/api_docs/python/tfa/seq2seq/tile_batch
   - title: TrainingSampler
     path: /addons/api_docs/python/tfa/seq2seq/TrainingSampler
-  - title: attention_wrapper
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/seq2seq/attention_wrapper
-  - title: basic_decoder
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/seq2seq/basic_decoder
-  - title: beam_search_decoder
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/seq2seq/beam_search_decoder
-    - title: attention_probs_from_attn_state
-      path: /addons/api_docs/python/tfa/seq2seq/beam_search_decoder/attention_probs_from_attn_state
-    - title: BeamSearchDecoderMixin
-      path: /addons/api_docs/python/tfa/seq2seq/beam_search_decoder/BeamSearchDecoderMixin
-    - title: get_attention_probs
-      path: /addons/api_docs/python/tfa/seq2seq/beam_search_decoder/get_attention_probs
-  - title: decoder
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/seq2seq/decoder
-  - title: loss
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/seq2seq/loss
-  - title: sampler
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/seq2seq/sampler
-    - title: bernoulli_sample
-      path: /addons/api_docs/python/tfa/seq2seq/sampler/bernoulli_sample
-    - title: categorical_sample
-      path: /addons/api_docs/python/tfa/seq2seq/sampler/categorical_sample
 - title: tfa.text
   section:
   - title: Overview
@@ -707,17 +359,21 @@ toc:
     path: /addons/api_docs/python/tfa/text/skip_gram_sample_with_text_vocab
   - title: viterbi_decode
     path: /addons/api_docs/python/tfa/text/viterbi_decode
-  - title: crf
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/text/crf
-    - title: CrfDecodeForwardRnnCell
-      path: /addons/api_docs/python/tfa/text/crf/CrfDecodeForwardRnnCell
-  - title: parse_time_op
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/text/parse_time_op
-  - title: skip_gram_ops
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/text/skip_gram_ops
+- title: tfa.types
+  section:
+  - title: Overview
+    path: /addons/api_docs/python/tfa/types
+  - title: AcceptableDTypes
+    path: /addons/api_docs/python/tfa/types/AcceptableDTypes
+  - title: Activation
+    path: /addons/api_docs/python/tfa/types/Activation
+  - title: Constraint
+    path: /addons/api_docs/python/tfa/types/Constraint
+  - title: FloatTensorLike
+    path: /addons/api_docs/python/tfa/types/FloatTensorLike
+  - title: Number
+    path: /addons/api_docs/python/tfa/types/Number
+  - title: Optimizer
+    path: /addons/api_docs/python/tfa/types/Optimizer
+  - title: TensorLike
+    path: /addons/api_docs/python/tfa/types/TensorLike

Diff gaussian_filter2d.md for example

diff --git a/home/windqaq/addons_api/tfa/image/gaussian_filter2d.md b/home/windqaq/new_addons_api/tfa/image/gaussian_filter2d.md
index ce6e66a..4f8678f 100644
--- a/home/windqaq/addons_api/tfa/image/gaussian_filter2d.md
+++ b/home/windqaq/new_addons_api/tfa/image/gaussian_filter2d.md
@@ -22,24 +22,16 @@ description: Perform Gaussian blur on image(s).

 Perform Gaussian blur on image(s).

-

-
View aliases

-

-Main aliases
-
`tfa.image.filters.gaussian_filter2d`

-

-

-

 @tf.function
 tfa.image.gaussian_filter2d(
-    image: tfa.image.filters.FloatTensorLike,
+    image: tfa.types.FloatTensorLike,
     filter_shape: Union[List[int], Tuple[int]] = [3, 3],
     sigma: Union[List[float], Tuple[float]] = 1.0,
     padding: str = 'REFLECT',
-    constant_values: tfa.image.color_ops.TensorLike = 0,
+    constant_values: tfa.types.TensorLike = 0,
     name: Optional[str] = None
-) -> tfa.image.filters.FloatTensorLike
+) -> tfa.types.FloatTensorLike

Type of change

[ ] Bug fix
[ ] New Tutorial
[x] Updated or additional documentation
[ ] Additional Testing
[ ] New Activation and the changes conform to the activation contribution guidelines
[ ] New Callback and the changes conform to the callback contribution guidelines
[ ] New Image addition and the changes conform to the image op contribution guidelines
[ ] New Layer and the changes conform to the layer contribution guidelines
[ ] New Loss and the changes conform to the loss contribution guidelines
[ ] New Metric and the changes conform to the metric contribution guidelines
[ ] New Optimizer and the changes conform to the optimizer contribution guidelines
[ ] New RNN Cell and the changes conform to the rnn contribution guidelines
[ ] New Seq2seq addition and the changes conform to the seq2seq contribution guidelines
[ ] New Text addition and the changes conform to the text op contribution guidelines

Checklist:

[x] I've properly formatted my code according to the guidelines

[ ] By running Black + Flake8
[ ] By running pre-commit hooks

[ ] This PR addresses an already submitted issue for TensorFlow Addons
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] This PR contains modifications to C++ custom-ops

How Has This Been Tested?
If you're adding a bugfix or new feature please describe the tests that you ran to verify your changes:
*

@MarkDaoust, Hi Mark, can you help review this when time allows? Thank you!

explicit_package_contents_filter

Good idea.
The main thing to remember with this setting is that the now the imports in your__init__.py are the definition of your public API. With this is's best not to put any code in the init files other than imports.
If that's what you want -> great. If ever something isn't showing up it's probably just missing an import.
If you've been test-running this could you post a git diff --stat of the output directory? this would make it clearwhat affect this has on the generated pages.

The main thing to remember with this setting is that the now the imports in your__init__.py are the definition of your public API. With this is's best not to put any code in the init files other than imports.

That's actually what I want. /cc @seanpmorgan @bhack for discussion.
Here is the diff file.
diff

 .../{addons_api => new_addons_api}/_redirects.yaml |  444 +-
 .../{addons_api => new_addons_api}/_toc.yaml       |  384 +-
 .../{addons_api => new_addons_api}/index.md        |  342 +-
 .../tfa/_api_cache.json                            | 6081 +-------------------
 .../tfa/activations/gelu.md                        |    2 +-
 .../tfa/activations/hardshrink.md                  |    6 +-
 .../tfa/activations/lisht.md                       |    2 +-
 .../tfa/activations/mish.md                        |    2 +-
 .../tfa/activations/rrelu.md                       |    6 +-
 .../tfa/activations/snake.md                       |    4 +-
 .../tfa/activations/softshrink.md                  |    6 +-
 .../tfa/activations/sparsemax.md                   |    2 +-
 .../tfa/activations/tanhshrink.md                  |    2 +-
 .../tfa/callbacks/AverageModelCheckpoint.md        |    8 -
 .../tfa/callbacks/TQDMProgressBar.md               |    8 -
 .../tfa/callbacks/TimeStopping.md                  |    8 -
 .../average_model_checkpoint.md => dev/null}       |   28 -
 .../tfa/callbacks/time_stopping.md => dev/null}    |   30 -
 .../callbacks/tqdm_progress_bar.md => dev/null}    |   30 -
 .../tfa/callbacks.md                               |    8 -
 .../tfa/image/adjust_hsv_in_yiq.md                 |   16 +-
 .../tfa/image/blend.md                             |   14 +-
 .../image/color_ops/equalize_image.md => dev/null} |   34 -
 .../color_ops/sharpness_image.md => dev/null}      |   34 -
 .../addons_api/tfa/image/color_ops.md => dev/null} |   43 -
 .../tfa/image/compose_ops.md => dev/null}          |   36 -
 .../tfa/image/connected_components.md              |    2 +-
 .../tfa/image/cutout.md                            |   16 +-
 .../tfa/image/cutout_ops.md => dev/null}           |   38 -
 .../tfa/image/dense_image_warp.md                  |    4 +-
 .../tfa/image/distance_transform.md => dev/null}   |   34 -
 .../tfa/image/distort_image_ops.md => dev/null}    |   38 -
 .../tfa/image/equalize.md                          |   10 +-
 .../tfa/image/euclidean_dist_transform.md          |   10 +-
 .../addons_api/tfa/image/filters.md => dev/null}   |   38 -
 .../tfa/image/gaussian_filter2d.md                 |   14 +-
 .../tfa/image/interpolate_bilinear.md              |    4 +-
 .../tfa/image/interpolate_spline.md                |    8 +-
 .../tfa/image/mean_filter2d.md                     |   12 +-
 .../tfa/image/median_filter2d.md                   |   12 +-
 .../tfa/image/random_cutout.md                     |   16 +-
 .../tfa/image/random_hsv_in_yiq.md                 |   20 +-
 .../tfa/image/resampler.md                         |   12 +-
 .../tfa/image/resampler_ops.md => dev/null}        |   30 -
 .../tfa/image/rotate.md                            |   12 +-
 .../tfa/image/sharpness.md                         |   12 +-
 .../tfa/image/shear_x.md                           |   14 +-
 .../tfa/image/shear_y.md                           |   14 +-
 .../tfa/image/sparse_image_warp.md                 |    8 +-
 .../tfa/image/transform.md                         |   12 +-
 .../null}                                          |   84 -
 .../compose_transforms.md => dev/null}             |   77 -
 .../flat_transforms_to_matrices.md => dev/null}    |   94 -
 .../matrices_to_flat_transforms.md => dev/null}    |   95 -
 .../tfa/image/transform_ops.md => dev/null}        |   48 -
 .../tfa/image/translate.md                         |   12 +-
 .../null}                                          |   76 -
 .../tfa/image/translate_ops.md => dev/null}        |   38 -
 .../tfa/image/translate_xy.md                      |   16 +-
 .../tfa/image/utils/from_4D_image.md => dev/null}  |   71 -
 .../tfa/image/utils/get_ndims.md => dev/null}      |   31 -
 .../tfa/image/utils/to_4D_image.md => dev/null}    |   64 -
 .../tfa/image/utils/unwrap.md => dev/null}         |   81 -
 .../tfa/image/utils/wrap.md => dev/null}           |   33 -
 .../addons_api/tfa/image/utils.md => dev/null}     |   38 -
 .../{addons_api => new_addons_api}/tfa/image.md    |   22 -
 .../tfa/layers/AdaptiveAveragePooling1D.md         |   10 -
 .../tfa/layers/AdaptiveAveragePooling2D.md         |   10 -
 .../tfa/layers/AdaptiveAveragePooling3D.md         |   10 -
 .../tfa/layers/AdaptiveMaxPooling1D.md             |   10 -
 .../tfa/layers/AdaptiveMaxPooling2D.md             |   10 -
 .../tfa/layers/AdaptiveMaxPooling3D.md             |   10 -
 .../tfa/layers/CorrelationCost.md                  |    8 -
 .../tfa/layers/ESN.md                              |   24 +-
 .../tfa/layers/FilterResponseNormalization.md      |   22 +-
 .../tfa/layers/GELU.md                             |    8 -
 .../tfa/layers/GroupNormalization.md               |   20 +-
 .../tfa/layers/InstanceNormalization.md            |    8 -
 .../tfa/layers/Maxout.md                           |    8 -
 .../tfa/layers/MultiHeadAttention.md               |    8 -
 .../tfa/layers/PoincareNormalize.md                |    8 -
 .../tfa/layers/PolynomialCrossing.md               |   16 +-
 .../tfa/layers/Snake.md                            |   10 +-
 .../tfa/layers/Sparsemax.md                        |    8 -
 .../tfa/layers/SpatialPyramidPooling2D.md          |    8 -
 .../tfa/layers/SpectralNormalization.md            |    8 -
 .../tfa/layers/TLU.md                              |   20 +-
 .../tfa/layers/WeightNormalization.md              |    8 -
 .../AdaptivePooling1D.md => dev/null}              | 1043 ----
 .../AdaptivePooling2D.md => dev/null}              | 1043 ----
 .../AdaptivePooling3D.md => dev/null}              | 1043 ----
 .../tfa/layers/adaptive_pooling.md => dev/null}    |   46 -
 .../addons_api/tfa/layers/esn.md => dev/null}      |   40 -
 .../addons_api/tfa/layers/gelu.md => dev/null}     |   30 -
 .../addons_api/tfa/layers/maxout.md => dev/null}   |   30 -
 .../tfa/layers/multihead_attention.md => dev/null} |   28 -
 .../tfa/layers/normalizations.md => dev/null}      |   32 -
 .../tfa/layers/optical_flow.md => dev/null}        |   30 -
 .../addons_api/tfa/layers/poincare.md => dev/null} |   30 -
 .../tfa/layers/polynomial.md => dev/null}          |   30 -
 .../addons_api/tfa/layers/snake.md => dev/null}    |   30 -
 .../tfa/layers/sparsemax.md => dev/null}           |   28 -
 .../layers/spatial_pyramid_pooling.md => dev/null} |   30 -
 .../layers/spectral_normalization.md => dev/null}  |   28 -
 .../addons_api/tfa/layers/tlu.md => dev/null}      |   30 -
 .../addons_api/tfa/layers/wrappers.md => dev/null} |   28 -
 .../{addons_api => new_addons_api}/tfa/layers.md   |   32 -
 .../tfa/losses/ContrastiveLoss.md                  |   10 +-
 .../tfa/losses/LiftedStructLoss.md                 |   10 +-
 .../tfa/losses/NpairsLoss.md                       |    8 -
 .../tfa/losses/NpairsMultilabelLoss.md             |    8 -
 .../tfa/losses/PinballLoss.md                      |   10 +-
 .../tfa/losses/SigmoidFocalCrossEntropy.md         |   12 +-
 .../tfa/losses/TripletHardLoss.md                  |   10 +-
 .../tfa/losses/TripletSemiHardLoss.md              |   10 +-
 .../tfa/losses/WeightedKappaLoss.md                |    8 -
 .../tfa/losses/contrastive.md => dev/null}         |   40 -
 .../tfa/losses/contrastive_loss.md                 |   14 +-
 .../tfa/losses/focal_loss.md => dev/null}          |   40 -
 .../tfa/losses/giou_loss.md                        |    4 +-
 .../tfa/losses/kappa_loss.md => dev/null}          |   34 -
 .../addons_api/tfa/losses/lifted.md => dev/null}   |   40 -
 .../tfa/losses/lifted_struct_loss.md               |   14 +-
 .../angular_distance.md => dev/null}               |   69 -
 .../pairwise_distance.md => dev/null}              |   77 -
 .../tfa/losses/metric_learning.md => dev/null}     |   36 -
 .../addons_api/tfa/losses/npairs.md => dev/null}   |   42 -
 .../tfa/losses/npairs_loss.md                      |   12 +-
 .../tfa/losses/npairs_multilabel_loss.md           |   12 +-
 .../tfa/losses/pinball_loss.md                     |   14 +-
 .../tfa/losses/quantiles.md => dev/null}           |   40 -
 .../tfa/losses/sigmoid_focal_crossentropy.md       |   16 +-
 .../tfa/losses/sparsemax_loss.md                   |    6 +-
 .../addons_api/tfa/losses/triplet.md => dev/null}  |   44 -
 .../tfa/losses/triplet_hard_loss.md                |   14 +-
 .../tfa/losses/triplet_semihard_loss.md            |   14 +-
 .../{addons_api => new_addons_api}/tfa/losses.md   |   18 -
 .../tfa/metrics/CohenKappa.md                      |   12 +-
 .../tfa/metrics/F1Score.md                         |   12 +-
 .../tfa/metrics/FBetaScore.md                      |   14 +-
 .../tfa/metrics/GeometricMean.md                   |   10 +-
 .../tfa/metrics/HammingLoss.md                     |   10 +-
 .../tfa/metrics/MatthewsCorrelationCoefficient.md  |   12 +-
 .../tfa/metrics/MeanMetricWrapper.md               |   10 +-
 .../tfa/metrics/MultiLabelConfusionMatrix.md       |   12 +-
 .../tfa/metrics/RSquare.md                         |   10 +-
 .../tfa/metrics/cohens_kappa.md => dev/null}       |   36 -
 .../tfa/metrics/f_scores.md => dev/null}           |   38 -
 .../tfa/metrics/geometric_mean.md => dev/null}     |   34 -
 .../hamming/hamming_loss_fn.md => dev/null}        |  128 -
 .../addons_api/tfa/metrics/hamming.md => dev/null} |   44 -
 .../tfa/metrics/hamming_distance.md                |   12 +-
 .../null}                                          |   36 -
 .../multilabel_confusion_matrix.md => dev/null}    |   36 -
 .../tfa/metrics/r_square.md => dev/null}           |   38 -
 .../sample_weight_shape_match.md => dev/null}      |   31 -
 .../addons_api/tfa/metrics/utils.md => dev/null}   |   38 -
 .../{addons_api => new_addons_api}/tfa/metrics.md  |   18 -
 .../tfa/optimizers/AdamW.md                        |   12 +-
 .../tfa/optimizers/AveragedOptimizerWrapper.md     |   10 +-
 .../tfa/optimizers/ConditionalGradient.md          |   10 +-
 .../tfa/optimizers/CyclicalLearningRate.md         |   10 +-
 .../optimizers/ExponentialCyclicalLearningRate.md  |   12 +-
 .../tfa/optimizers/LAMB.md                         |   16 +-
 .../tfa/optimizers/LazyAdam.md                     |   14 +-
 .../tfa/optimizers/Lookahead.md                    |   12 +-
 .../tfa/optimizers/MovingAverage.md                |   12 +-
 .../tfa/optimizers/NovoGrad.md                     |   16 +-
 .../tfa/optimizers/ProximalAdagrad.md              |    8 -
 .../tfa/optimizers/RectifiedAdam.md                |   20 +-
 .../tfa/optimizers/SGDW.md                         |   10 -
 .../tfa/optimizers/SWA.md                          |   10 +-
 .../optimizers/Triangular2CyclicalLearningRate.md  |   10 +-
 .../optimizers/TriangularCyclicalLearningRate.md   |   10 +-
 .../tfa/optimizers/Yogi.md                         |   20 +-
 .../tfa/optimizers/average_wrapper.md => dev/null} |   28 -
 .../conditional_gradient.md => dev/null}           |   34 -
 .../cyclical_learning_rate.md => dev/null}         |   40 -
 .../extend_with_decoupled_weight_decay.md          |    8 -
 .../addons_api/tfa/optimizers/lamb.md => dev/null} |   36 -
 .../tfa/optimizers/lazy_adam.md => dev/null}       |   38 -
 .../tfa/optimizers/lookahead.md => dev/null}       |   28 -
 .../tfa/optimizers/moving_average.md => dev/null}  |   28 -
 .../tfa/optimizers/novograd.md => dev/null}        |   34 -
 .../optimizers/proximal_adagrad.md => dev/null}    |   34 -
 .../tfa/optimizers/rectified_adam.md => dev/null}  |   34 -
 .../stochastic_weight_averaging.md => dev/null}    |   37 -
 .../DecoupledWeightDecayExtension.md => dev/null}  |  345 --
 .../weight_decay_optimizers.md => dev/null}        |   42 -
 .../addons_api/tfa/optimizers/yogi.md => dev/null} |   40 -
 .../tfa/optimizers.md                              |   28 -
 .../tfa/options/warn_fallback.md => dev/null}      |   31 -
 .../windqaq/addons_api/tfa/options.md => dev/null} |   34 -
 .../register_custom_kernels.md => dev/null}        |   29 -
 .../register_keras_objects.md => dev/null}         |   29 -
 .../addons_api/tfa/register.md => dev/null}        |   32 -
 .../tfa}/register_all.md                           |   14 +-
 .../tfa/rnn/ESNCell.md                             |   16 +-
 .../tfa/rnn/LayerNormLSTMCell.md                   |   42 +-
 .../tfa/rnn/LayerNormSimpleRNNCell.md              |   42 +-
 .../tfa/rnn/NASCell.md                             |   18 +-
 .../tfa/rnn/PeepholeLSTMCell.md                    |    8 -
 .../addons_api/tfa/rnn/esn_cell.md => dev/null}    |   36 -
 .../tfa/rnn/layer_norm_lstm_cell.md => dev/null}   |   44 -
 .../rnn/layer_norm_simple_rnn_cell.md => dev/null} |   44 -
 .../addons_api/tfa/rnn/nas_cell.md => dev/null}    |   38 -
 .../tfa/rnn/peephole_lstm_cell.md => dev/null}     |   30 -
 .../{addons_api => new_addons_api}/tfa/rnn.md      |   12 -
 .../tfa/seq2seq/AttentionMechanism.md              |    8 -
 .../tfa/seq2seq/AttentionWrapper.md                |    8 -
 .../tfa/seq2seq/AttentionWrapperState.md           |    8 -
 .../tfa/seq2seq/BahdanauAttention.md               |   14 +-
 .../tfa/seq2seq/BahdanauMonotonicAttention.md      |   18 +-
 .../tfa/seq2seq/BaseDecoder.md                     |    8 -
 .../tfa/seq2seq/BasicDecoder.md                    |    8 -
 .../tfa/seq2seq/BasicDecoderOutput.md              |    8 -
 .../tfa/seq2seq/BeamSearchDecoder.md               |   14 +-
 .../tfa/seq2seq/BeamSearchDecoderOutput.md         |    8 -
 .../tfa/seq2seq/BeamSearchDecoderState.md          |    8 -
 .../tfa/seq2seq/CustomSampler.md                   |   12 +-
 .../tfa/seq2seq/Decoder.md                         |    8 -
 .../tfa/seq2seq/FinalBeamSearchDecoderOutput.md    |    8 -
 .../tfa/seq2seq/GreedyEmbeddingSampler.md          |    8 -
 .../tfa/seq2seq/InferenceSampler.md                |   12 +-
 .../tfa/seq2seq/LuongAttention.md                  |   12 +-
 .../tfa/seq2seq/LuongMonotonicAttention.md         |   16 +-
 .../tfa/seq2seq/SampleEmbeddingSampler.md          |    8 -
 .../tfa/seq2seq/Sampler.md                         |    8 -
 .../seq2seq/ScheduledEmbeddingTrainingSampler.md   |   10 +-
 .../tfa/seq2seq/ScheduledOutputTrainingSampler.md  |   10 +-
 .../tfa/seq2seq/SequenceLoss.md                    |    8 -
 .../tfa/seq2seq/TrainingSampler.md                 |    8 -
 .../tfa/seq2seq/attention_wrapper.md => dev/null}  |   62 -
 .../tfa/seq2seq/basic_decoder.md => dev/null}      |   32 -
 .../BeamSearchDecoderMixin.md => dev/null}         |  318 -
 .../null}                                          |   66 -
 .../get_attention_probs.md => dev/null}            |   92 -
 .../seq2seq/beam_search_decoder.md => dev/null}    |   58 -
 .../addons_api/tfa/seq2seq/decoder.md => dev/null} |   40 -
 .../tfa/seq2seq/dynamic_decode.md                  |    8 -
 .../tfa/seq2seq/gather_tree.md                     |   16 +-
 .../tfa/seq2seq/gather_tree_from_array.md          |   14 +-
 .../tfa/seq2seq/hardmax.md                         |   10 +-
 .../addons_api/tfa/seq2seq/loss.md => dev/null}    |   38 -
 .../tfa/seq2seq/monotonic_attention.md             |   12 +-
 .../tfa/seq2seq/safe_cumprod.md                    |   10 +-
 .../sampler/bernoulli_sample.md => dev/null}       |   33 -
 .../sampler/categorical_sample.md => dev/null}     |   33 -
 .../addons_api/tfa/seq2seq/sampler.md => dev/null} |   56 -
 .../tfa/seq2seq/sequence_loss.md                   |   14 +-
 .../tfa/seq2seq/tile_batch.md                      |   10 +-
 .../{addons_api => new_addons_api}/tfa/seq2seq.md  |   14 -
 .../crf/CrfDecodeForwardRnnCell.md => dev/null}    | 1053 ----
 .../addons_api/tfa/text/crf.md => dev/null}        |   56 -
 .../tfa/text/crf_binary_score.md                   |   14 +-
 .../tfa/text/crf_decode.md                         |   14 +-
 .../tfa/text/crf_decode_backward.md                |   12 +-
 .../tfa/text/crf_decode_forward.md                 |   16 +-
 .../tfa/text/crf_forward.md                        |   16 +-
 .../tfa/text/crf_log_likelihood.md                 |   14 +-
 .../tfa/text/crf_log_norm.md                       |   14 +-
 .../tfa/text/crf_multitag_sequence_score.md        |   16 +-
 .../tfa/text/crf_sequence_score.md                 |   16 +-
 .../tfa/text/crf_unary_score.md                    |   14 +-
 .../tfa/text/parse_time.md                         |    8 -
 .../tfa/text/parse_time_op.md => dev/null}         |   34 -
 .../tfa/text/skip_gram_ops.md => dev/null}         |   40 -
 .../tfa/text/skip_gram_sample.md                   |   18 +-
 .../tfa/text/skip_gram_sample_with_text_vocab.md   |   22 +-
 .../tfa/text/viterbi_decode.md                     |   12 +-
 .../{addons_api => new_addons_api}/tfa/text.md     |    8 -
 .../tfa/types}/AcceptableDTypes.md                 |    4 +-
 .../esn => new_addons_api/tfa/types}/Activation.md |    4 +-
 .../tfa/types/Constraint.md}                       |    6 +-
 .../tfa/types}/FloatTensorLike.md                  |    4 +-
 .../tfa/types}/Number.md                           |    4 +-
 .../windqaq/new_addons_api/tfa/types/Optimizer.md} |   32 +
 .../tfa/types}/TensorLike.md                       |    4 +-
 .../windqaq/new_addons_api/tfa/types.md}           |   46 +
 .../windqaq/{addons_api => new_addons_api}/tfa.md  |    8 +-
 280 files changed, 485 insertions(+), 17753 deletions(-)

SGTM.
Sorry that diff's not as clear as I was hoping... if you diff the git diff the generated _toc.yaml  that might show more clearly what's changed.

SGTM.
Sorry that diff's not as clear as I was hoping... if you diff the git diff the generated _toc.yaml that might show more clearly what's changed.

Here it is.
Diff TOC

diff --git a/home/windqaq/addons_api/_toc.yaml b/home/windqaq/new_addons_api/_toc.yaml
index 7c9c58a..45ee1f7 100644
--- a/home/windqaq/addons_api/_toc.yaml
+++ b/home/windqaq/new_addons_api/_toc.yaml
@@ -3,6 +3,8 @@ toc:
   section:
   - title: Overview
     path: /addons/api_docs/python/tfa
+  - title: register_all
+    path: /addons/api_docs/python/tfa/register_all
 - title: tfa.activations
   section:
   - title: Overview
@@ -35,18 +37,6 @@ toc:
     path: /addons/api_docs/python/tfa/callbacks/TimeStopping
   - title: TQDMProgressBar
     path: /addons/api_docs/python/tfa/callbacks/TQDMProgressBar
-  - title: average_model_checkpoint
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/callbacks/average_model_checkpoint
-  - title: time_stopping
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/callbacks/time_stopping
-  - title: tqdm_progress_bar
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/callbacks/tqdm_progress_bar
 - title: tfa.image
   section:
   - title: Overview
@@ -97,76 +87,6 @@ toc:
     path: /addons/api_docs/python/tfa/image/translate
   - title: translate_xy
     path: /addons/api_docs/python/tfa/image/translate_xy
-  - title: color_ops
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/color_ops
-    - title: equalize_image
-      path: /addons/api_docs/python/tfa/image/color_ops/equalize_image
-    - title: Number
-      path: /addons/api_docs/python/tfa/image/color_ops/Number
-    - title: sharpness_image
-      path: /addons/api_docs/python/tfa/image/color_ops/sharpness_image
-    - title: TensorLike
-      path: /addons/api_docs/python/tfa/image/color_ops/TensorLike
-  - title: compose_ops
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/compose_ops
-  - title: cutout_ops
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/cutout_ops
-  - title: distance_transform
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/distance_transform
-  - title: distort_image_ops
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/distort_image_ops
-  - title: filters
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/filters
-    - title: FloatTensorLike
-      path: /addons/api_docs/python/tfa/image/filters/FloatTensorLike
-  - title: resampler_ops
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/resampler_ops
-  - title: transform_ops
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/transform_ops
-    - title: angles_to_projective_transforms
-      path: /addons/api_docs/python/tfa/image/transform_ops/angles_to_projective_transforms
-    - title: compose_transforms
-      path: /addons/api_docs/python/tfa/image/transform_ops/compose_transforms
-    - title: flat_transforms_to_matrices
-      path: /addons/api_docs/python/tfa/image/transform_ops/flat_transforms_to_matrices
-    - title: matrices_to_flat_transforms
-      path: /addons/api_docs/python/tfa/image/transform_ops/matrices_to_flat_transforms
-  - title: translate_ops
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/translate_ops
-    - title: translations_to_projective_transforms
-      path: /addons/api_docs/python/tfa/image/translate_ops/translations_to_projective_transforms
-  - title: utils
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/image/utils
-    - title: from_4D_image
-      path: /addons/api_docs/python/tfa/image/utils/from_4D_image
-    - title: get_ndims
-      path: /addons/api_docs/python/tfa/image/utils/get_ndims
-    - title: to_4D_image
-      path: /addons/api_docs/python/tfa/image/utils/to_4D_image
-    - title: unwrap
-      path: /addons/api_docs/python/tfa/image/utils/unwrap
-    - title: wrap
-      path: /addons/api_docs/python/tfa/image/utils/wrap
 - title: tfa.layers
   section:
   - title: Overview
@@ -215,76 +135,6 @@ toc:
     path: /addons/api_docs/python/tfa/layers/TLU
   - title: WeightNormalization
     path: /addons/api_docs/python/tfa/layers/WeightNormalization
-  - title: adaptive_pooling
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/adaptive_pooling
-    - title: AdaptivePooling1D
-      path: /addons/api_docs/python/tfa/layers/adaptive_pooling/AdaptivePooling1D
-    - title: AdaptivePooling2D
-      path: /addons/api_docs/python/tfa/layers/adaptive_pooling/AdaptivePooling2D
-    - title: AdaptivePooling3D
-      path: /addons/api_docs/python/tfa/layers/adaptive_pooling/AdaptivePooling3D
-  - title: esn
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/esn
-    - title: Activation
-      path: /addons/api_docs/python/tfa/layers/esn/Activation
-    - title: Initializer
-      path: /addons/api_docs/python/tfa/layers/esn/Initializer
-  - title: gelu
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/gelu
-  - title: maxout
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/maxout
-  - title: multihead_attention
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/multihead_attention
-  - title: normalizations
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/normalizations
-  - title: optical_flow
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/optical_flow
-  - title: poincare
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/poincare
-  - title: polynomial
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/polynomial
-  - title: snake
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/snake
-  - title: sparsemax
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/sparsemax
-  - title: spatial_pyramid_pooling
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/spatial_pyramid_pooling
-  - title: spectral_normalization
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/spectral_normalization
-  - title: tlu
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/tlu
-  - title: wrappers
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/layers/wrappers
 - title: tfa.losses
   section:
   - title: Overview
@@ -331,42 +181,6 @@ toc:
     path: /addons/api_docs/python/tfa/losses/triplet_semihard_loss
   - title: WeightedKappaLoss
     path: /addons/api_docs/python/tfa/losses/WeightedKappaLoss
-  - title: contrastive
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/losses/contrastive
-  - title: focal_loss
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/losses/focal_loss
-  - title: kappa_loss
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/losses/kappa_loss
-  - title: lifted
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/losses/lifted
-  - title: metric_learning
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/losses/metric_learning
-    - title: angular_distance
-      path: /addons/api_docs/python/tfa/losses/metric_learning/angular_distance
-    - title: pairwise_distance
-      path: /addons/api_docs/python/tfa/losses/metric_learning/pairwise_distance
-  - title: npairs
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/losses/npairs
-  - title: quantiles
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/losses/quantiles
-  - title: triplet
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/losses/triplet
 - title: tfa.metrics
   section:
   - title: Overview
@@ -391,44 +205,6 @@ toc:
     path: /addons/api_docs/python/tfa/metrics/MultiLabelConfusionMatrix
   - title: RSquare
     path: /addons/api_docs/python/tfa/metrics/RSquare
-  - title: cohens_kappa
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/metrics/cohens_kappa
-    - title: AcceptableDTypes
-      path: /addons/api_docs/python/tfa/metrics/cohens_kappa/AcceptableDTypes
-  - title: f_scores
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/metrics/f_scores
-  - title: geometric_mean
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/metrics/geometric_mean
-  - title: hamming
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/metrics/hamming
-    - title: hamming_loss_fn
-      path: /addons/api_docs/python/tfa/metrics/hamming/hamming_loss_fn
-  - title: matthews_correlation_coefficient
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/metrics/matthews_correlation_coefficient
-  - title: multilabel_confusion_matrix
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/metrics/multilabel_confusion_matrix
-  - title: r_square
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/metrics/r_square
-  - title: utils
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/metrics/utils
-    - title: sample_weight_shape_match
-      path: /addons/api_docs/python/tfa/metrics/utils/sample_weight_shape_match
 - title: tfa.optimizers
   section:
   - title: Overview
@@ -469,76 +245,6 @@ toc:
     path: /addons/api_docs/python/tfa/optimizers/TriangularCyclicalLearningRate
   - title: Yogi
     path: /addons/api_docs/python/tfa/optimizers/Yogi
-  - title: average_wrapper
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/average_wrapper
-  - title: conditional_gradient
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/conditional_gradient
-  - title: cyclical_learning_rate
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/cyclical_learning_rate
-  - title: lamb
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/lamb
-  - title: lazy_adam
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/lazy_adam
-  - title: lookahead
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/lookahead
-  - title: moving_average
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/moving_average
-  - title: novograd
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/novograd
-  - title: proximal_adagrad
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/proximal_adagrad
-  - title: rectified_adam
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/rectified_adam
-  - title: stochastic_weight_averaging
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/stochastic_weight_averaging
-  - title: weight_decay_optimizers
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/weight_decay_optimizers
-    - title: DecoupledWeightDecayExtension
-      path: /addons/api_docs/python/tfa/optimizers/weight_decay_optimizers/DecoupledWeightDecayExtension
-  - title: yogi
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/optimizers/yogi
-- title: tfa.options
-  section:
-  - title: Overview
-    path: /addons/api_docs/python/tfa/options
-  - title: warn_fallback
-    path: /addons/api_docs/python/tfa/options/warn_fallback
-- title: tfa.register
-  section:
-  - title: Overview
-    path: /addons/api_docs/python/tfa/register
-  - title: register_all
-    path: /addons/api_docs/python/tfa/register/register_all
-  - title: register_custom_kernels
-    path: /addons/api_docs/python/tfa/register/register_custom_kernels
-  - title: register_keras_objects
-    path: /addons/api_docs/python/tfa/register/register_keras_objects
 - title: tfa.rnn
   section:
   - title: Overview
@@ -553,26 +259,6 @@ toc:
     path: /addons/api_docs/python/tfa/rnn/NASCell
   - title: PeepholeLSTMCell
     path: /addons/api_docs/python/tfa/rnn/PeepholeLSTMCell
-  - title: esn_cell
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/rnn/esn_cell
-  - title: layer_norm_lstm_cell
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/rnn/layer_norm_lstm_cell
-  - title: layer_norm_simple_rnn_cell
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/rnn/layer_norm_simple_rnn_cell
-  - title: nas_cell
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/rnn/nas_cell
-  - title: peephole_lstm_cell
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/rnn/peephole_lstm_cell
 - title: tfa.seq2seq
   section:
   - title: Overview
@@ -641,40 +327,6 @@ toc:
     path: /addons/api_docs/python/tfa/seq2seq/tile_batch
   - title: TrainingSampler
     path: /addons/api_docs/python/tfa/seq2seq/TrainingSampler
-  - title: attention_wrapper
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/seq2seq/attention_wrapper
-  - title: basic_decoder
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/seq2seq/basic_decoder
-  - title: beam_search_decoder
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/seq2seq/beam_search_decoder
-    - title: attention_probs_from_attn_state
-      path: /addons/api_docs/python/tfa/seq2seq/beam_search_decoder/attention_probs_from_attn_state
-    - title: BeamSearchDecoderMixin
-      path: /addons/api_docs/python/tfa/seq2seq/beam_search_decoder/BeamSearchDecoderMixin
-    - title: get_attention_probs
-      path: /addons/api_docs/python/tfa/seq2seq/beam_search_decoder/get_attention_probs
-  - title: decoder
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/seq2seq/decoder
-  - title: loss
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/seq2seq/loss
-  - title: sampler
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/seq2seq/sampler
-    - title: bernoulli_sample
-      path: /addons/api_docs/python/tfa/seq2seq/sampler/bernoulli_sample
-    - title: categorical_sample
-      path: /addons/api_docs/python/tfa/seq2seq/sampler/categorical_sample
 - title: tfa.text
   section:
   - title: Overview
@@ -707,17 +359,21 @@ toc:
     path: /addons/api_docs/python/tfa/text/skip_gram_sample_with_text_vocab
   - title: viterbi_decode
     path: /addons/api_docs/python/tfa/text/viterbi_decode
-  - title: crf
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/text/crf
-    - title: CrfDecodeForwardRnnCell
-      path: /addons/api_docs/python/tfa/text/crf/CrfDecodeForwardRnnCell
-  - title: parse_time_op
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/text/parse_time_op
-  - title: skip_gram_ops
-    section:
-    - title: Overview
-      path: /addons/api_docs/python/tfa/text/skip_gram_ops
+- title: tfa.types
+  section:
+  - title: Overview
+    path: /addons/api_docs/python/tfa/types
+  - title: AcceptableDTypes
+    path: /addons/api_docs/python/tfa/types/AcceptableDTypes
+  - title: Activation
+    path: /addons/api_docs/python/tfa/types/Activation
+  - title: Constraint
+    path: /addons/api_docs/python/tfa/types/Constraint
+  - title: FloatTensorLike
+    path: /addons/api_docs/python/tfa/types/FloatTensorLike
+  - title: Number
+    path: /addons/api_docs/python/tfa/types/Number
+  - title: Optimizer
+    path: /addons/api_docs/python/tfa/types/Optimizer
+  - title: TensorLike
+    path: /addons/api_docs/python/tfa/types/TensorLike

Cool thanks,
So this looks mostly like a good change. But there are a few objects that will be dropped from the docs, many of these look like they are meant to be public, so if we are making this change you will need to add the missing imports to get the ones that are meant to be public to show up in the generated docs.
tfa/image/color_ops
image/filters/FloatTensorLike
tfa/image/transform_ops
tfa/image/translate_ops/translations_to_projective_transforms
tfa/image/utils
tfa/losses/metric_learning
tfa/metrics/hamming/hamming_loss_fn
tfa/metrics/utils/sample_weight_shape_match
tfa/optimizers/weight_decay_optimizers/DecoupledWeightDecayExtension
tfa/options/warn_fallback
tfa/register
tfa/seq2seq/beam_search_decoder
tfa/seq2seq/sampler
tfa/text/crf

We could use some feedback from the addons team here to indicate which of these are meant for public consumption.

Cool thanks,
So this looks mostly like a good change. But there are a few objects that will be dropped from the docs, many of these look like they are meant to be public, so if we are making this change you will need to add the missing imports to get the ones that are meant to be public to show up in the generated docs.
tfa/image/color_ops
image/filters/FloatTensorLike
tfa/image/transform_ops
tfa/image/translate_ops/translations_to_projective_transforms
tfa/image/utils
tfa/losses/metric_learning
tfa/metrics/hamming/hamming_loss_fn
tfa/metrics/utils/sample_weight_shape_match
tfa/optimizers/weight_decay_optimizers/DecoupledWeightDecayExtension
tfa/options/warn_fallback
tfa/register
tfa/seq2seq/beam_search_decoder
tfa/seq2seq/sampler
tfa/text/crf

We could use some feedback from the addons team here to indicate which of these are meant for public consumption.

Yep, some modules should be public. We'll clean it up and discuss it. Thank you!

@tensorflow/sig-addons-maintainers I have listed all modules to be discussed down below. Any comment on this? Thank you!

tfa/image/color_ops

Drop: equalize and sharpness already exported.

image/filters/FloatTensorLike

Drop: this is just the alias of tfa.utils.types.*

tfa/image/transform_ops

Expose

compose_transforms: Useful utility when users want to compose many transformation.
angles_to_projective_transforms: Convert angles to transformation matrix.

tfa/image/translate_ops/translations_to_projective_transforms

Expose

translations_to_projective_transforms: Convert translations to transformation matrix.

tfa/image/utils

Drop: internal utility

tfa/losses/metric_learning

Drop? People can use different distance through distance argument for triplet loss.

tfa/metrics/hamming/hamming_loss_fn

Expose? I think it's better to name this hamming_loss instead of hamming_loss_fn for consistency.

tfa/metrics/utils/sample_weight_shape_match

Drop: internal utility

tfa/optimizers/weight_decay_optimizers/DecoupledWeightDecayExtension

Expose: this is the wrapper of weight decay optimizer.

tfa/options/warn_fallback

Drop: internal use.

tfa/register

Drop: tfa.register_all() has already been exposed.

tfa/seq2seq/beam_search_decoder

Drop: main functions/classes have already been exposed to tfa.seq2seq.

tfa/seq2seq/sampler

Drop: main functions/classes have already been exposed to tfa.seq2seq.

tfa/text/crf:

Expose? maybe export tfa.text.crf.CrfDecodeForwardRnnCell as tfa.text.CrfDecodeForwardRnnCell?

Hmm, team tagging is not so useful. @seanpmorgan @bhack Can you review the comments above when time allows, I would like to move this forward. Thank you!

@WindQAQ It is the risk of https://en.m.wikipedia.org/wiki/Bystander_effect :smile:
Can you quickly explain what is this?

@WindQAQ It is the risk of https://en.m.wikipedia.org/wiki/Bystander_effect 😄

Haha, it's not a good effect actually 😆

Can you quickly explain what is this?

Okay, so the problem is orignated in https://github.com/tensorflow/addons/issues/2132.
In short, decide the public API in https://github.com/tensorflow/addons/pull/2162#issuecomment-696876373
TL;DR
Changes:

Fixes #2132. The exposed namespace aligns to the one of core TF: tf.types.experiemental.* vs tfa.types.*.
Use explicit_package_contents_filter to filter out meaningless endpoints. This also leads to the fact that only modules/functions/classes imported in __init__.py will be generated publicly. https://github.com/tensorflow/addons/pull/2162#issuecomment-696876373 lists all modules that I thought to be dropped/exposed after this PR. I will expose them according to the review in this PR as well.
Pin to newest tensorflow_docs version.

I think Is ok. I don't know much about CRF and seq2seq.

@tensorflow/sig-addons-maintainers I have listed all modules to be discussed down below. Any comment on this? Thank you!

tfa/image/color_ops

Drop: equalize and sharpness already exported.

image/filters/FloatTensorLike

Drop: this is just the alias of tfa.utils.types.*

tfa/image/transform_ops

Expose

compose_transforms: Useful utility when users want to compose many transformation.
angles_to_projective_transforms: Convert angles to transformation matrix.

tfa/image/translate_ops/translations_to_projective_transforms

Expose

translations_to_projective_transforms: Convert translations to transformation matrix.

tfa/image/utils

Drop: internal utility

tfa/losses/metric_learning

Drop? People can use different distance through distance argument for triplet loss.

tfa/metrics/hamming/hamming_loss_fn

Expose? I think it's better to name this hamming_loss instead of hamming_loss_fn for consistency.

tfa/metrics/utils/sample_weight_shape_match

Drop: internal utility

tfa/optimizers/weight_decay_optimizers/DecoupledWeightDecayExtension

Expose: this is the wrapper of weight decay optimizer.

tfa/options/warn_fallback

Drop: internal use.

tfa/register

Drop: tfa.register_all() has already been exposed.

tfa/seq2seq/beam_search_decoder

Drop: main functions/classes have already been exposed to tfa.seq2seq.

tfa/seq2seq/sampler

Drop: main functions/classes have already been exposed to tfa.seq2seq.

tfa/text/crf:

Expose? maybe export tfa.text.crf.CrfDecodeForwardRnnCell as tfa.text.CrfDecodeForwardRnnCell?

Agree with all of the above.... only thing I would change would be to expose all public functions in text.crf as I'm not sure of the user adoption and understanding there. One other thought:

Expose hamming_loss_fn and we can update the name in another PR (likely need to alias the old name though for backwards compat)

Sorry for the delay forgot to get around to this.

@WindQAQ just checking do you think you'll be able to clean up these or would you like someone to pick it up from here? No major rush but was thinking we should get this fixed up prior to Addons pinned to TF2.4

@seanpmorgan Sure, I'll do it in the weekend. Sorry that I'm busy with my course recently 😭

Hi @seanpmorgan @bhack, I've updated the PR. The updated API is here tfa_api.zip.
Let me now if there is any change needed!

authors: MarkDaoust WindQAQ bhack seanpmorgan

repo: tensorflow/addons


text:
Frozen weights

Hi, I am following the MNIST Keras tutorial. My goal is to implement a random projection layer by initializing the weights of a dense layer to a sparse random matrix.
Eventually, I want to be able to freeze the weights which are initially zero. However, I can't seem to get any layer at all, or even the model, to have frozen weights. I have tried applying both tf.stop_gradient to the weight tensor and setting the trainable parameter of my Keras layers to false. However, the parameters are still being trained.
Is there any advice to go about solving this problem? I guess my issue involves the train.py script.

This type of question is best asked on StackOverflow so others can help provide an answer: https://stackoverflow.com/search?q=cleverhans
In the meanwhile, I would suggest looking at the var_list argument of the train function. I'll close this issue but feel free to discuss further on StackOverflow unless you'd like to make a feature request or report a bug. Thank you!

source: gharchive/issue

repo: tensorflow/cleverhans

license: MIT

license_type: permissive

license_source: github-api


text:
cannot load the datasets

when i run:
import tensorflow_datasets as tfds
mnist_train = tfds.load(name="mnist", split="train")
wrong occured like this:
2020-08-20 09:52:45.879679: W tensorflow/core/platform/cloud/google_auth_provider.cc:178] All attempts to get a Google authentication bearer token failed, returning an empty token. Retrieving token from files failed with "Not found: Could not locate the credentials file.". Retrieving token from GCE failed with "Aborted: All 10 retry attempts failed. The last failure: Unavailable: Error executing an HTTP request: libcurl code 6 meaning 'Couldn't resolve host name', error details: Couldn't resolve host 'metadata'".
2020-08-20 09:53:46.902189: E tensorflow/core/platform/cloud/curl_http_request.cc:596] The transmission  of request 0x56396edb1ff0 (URI: https://www.googleapis.com/storage/v1/b/tfds-data/o/dataset_info%2Fmnist%2F3.0.1?fields=size%2Cgeneration%2Cupdated) has been stuck at 0 of 0 bytes for 61 seconds and will be aborted. CURL timing information: lookup time: 0.038829 (No error), connect time: 0 (No error), pre-transfer time: 0 (No error), start-transfer time: 0 (No error)
2020-08-20 09:54:48.722926: E tensorflow/core/platform/cloud/curl_http_request.cc:596] The transmission  of request 0x56396eeb1ef0 (URI: https://www.googleapis.com/storage/v1/b/tfds-data/o/dataset_info%2Fmnist%2F3.0.1?fields=size%2Cgeneration%2Cupdated) has been stuck at 0 of 0 bytes for 61 seconds and will be aborted. CURL timing information: lookup time: 0.070107 (No error), connect time: 0 (No error), pre-transfer time: 0 (No error), start-transfer time: 0 (No error)
2020-08-20 09:55:50.452114: E tensorflow/core/platform/cloud/curl_http_request.cc:596] The transmission  of request 0x56396edee160 (URI: https://www.googleapis.com/storage/v1/b/tfds-data/o/dataset_info%2Fmnist%2F3.0.1?fields=size%2Cgeneration%2Cupdated) has been stuck at 0 of 0 bytes for 61 seconds and will be aborted. CURL timing information: lookup time: 0.039491 (No error), connect time: 0 (No error), pre-transfer time: 0 (No error), start-transfer time: 0 (No error)
2020-08-20 09:56:57.020900: E tensorflow/core/platform/cloud/curl_http_request.cc:596] The transmission  of request 0x56396efbecd0 (URI: https://www.googleapis.com/storage/v1/b/tfds-data/o/dataset_info%2Fmnist%2F3.0.1?fields=size%2Cgeneration%2Cupdated) has been stuck at 0 of 0 bytes for 61 seconds and will be aborted. CURL timing information: lookup time: 5.08334 (No error), connect time: 0 (No error), pre-transfer time: 0 (No error), start-transfer time: 0 (No error)
2020-08-20 09:57:59.414296: E tensorflow/core/platform/cloud/curl_http_request.cc:596] The transmission  of request 0x56396f20f780 (URI: https://www.googleapis.com/storage/v1/b/tfds-data/o/dataset_info%2Fmnist%2F3.0.1?fields=size%2Cgeneration%2Cupdated) has been stuck at 0 of 0 bytes for 61 seconds and will be aborted. CURL timing information: lookup time: 0.039584 (No error), connect time: 0 (No error), pre-transfer time: 0 (No error), start-transfer time: 0 (No error)
i want to know how can i solve it , thank you

I  solve  it with https://github.com/tensorflow/datasets/issues/2182, downgrade tensorflow-datases to 3.1.0. Hope it works for you.

thank you so much

authors: angelawhitebear zprhhs


text:
zh-CN: translate graphs.ipynb

translate https://github.com/tensorflow/tensorboard/blob/master/docs/graphs.ipynb

@lamberta Hi, why the @tfdocsbot can not trigger the reviewer to review?

thanks for the heads up, @ccw1996
tfdocsbot should be up and running now on the next pull request

Here's my tfdocs impersonation until then:
Reviewers added, please take a look.
@kuri-leo, @JayYip, @yantaozhao, @loveunk, @wind2esg, @tigerneil, @MofiiTech, @gaoljhy, @Mr-Linus, @flopsySong, @echosun1996
When your review is finished, approve the pull request or include "LGTM" in your comment.
Preview and run these notebook edits with Google Colab:

site/zh-cn/tutorials/images/segmentation.ipynb

Notebook diffs available on ReviewNB.com.

source: gharchive/pull-request


text:
Convert a LazyTensorTrace into a TFFunction.

This PR introduces the functionality for the following conversions:
LazyTensorTrace  -> TFGraph -> TFFunction

@dan-zheng and I are at a conference. Perhaps @marcrasi can help with a readability review?

@dan-zheng and I are at a conference. Perhaps @marcrasi can help with a readability review?

Sounds good, Richard.

authors: bgogul rxwei

repo: tensorflow/swift-apis


text:
Memory Overhead/Leak in Android lib

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04):  Nexus 6p, Android v7.1.2
TensorFlow installed from (source or binary): source
TensorFlow version (use command below):  1.2.0-rc2
Python version: 2.7.10
Bazel version (if compiling from source): 0.4.5-homebrew
CUDA/cuDNN version: N/A
GPU model and memory:
Exact command to reproduce:
-- Selective Headers: bazel build -c opt --copt="-DSELECTIVE_REGISTRATION" --copt="-DSUPPORT_SELECTIVE_REGISTRATION" //tensorflow/contrib/android:libtensorflow_inference.so --crosstool_top=//external:android/crosstool --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --cpu=armeabi-v7a
-- Added tensorflow/core/kernels/random_shuffle_queue_op.cc and tensorflow/core/kernels/random_shuffle_op.cc to tf_op_files.txt file
-- Removed unused nodes: bazel build tensorflow/tools/graph_transforms:transform_graph bazel-bin/tensorflow/tools/graph_transforms/transform_graph \ --in_graph=model.pb \ --out_graph=optimized_model.pb \ --inputs='input' \ --outputs='output' \ --transforms=' strip_unused_nodes(type=float, shape="1,299,299,3")'

Describe the problem
The Tensorflow Android library is using a lot more memory than I expected. It almost seems like it's maintaining a reference to all input arrays, as memory usage balloons the longer the model is used.
Here is an example of the memory usage with feed/run/fetch commented out (source code below):

Here is the same timeframe, with the only difference being that feed/run/fetch is enabled:

Memory usage is over three times worse. The longer I leave the model running, the more memory usage increases (it eventually gets to 110 mb).
The below method is being called at a rate of 4.419011933 per sec (i.e. it's processing 4.412 input arrays per second), where each input array is of size 96*96*3 (27648).
This is being run on a Nexus 6p, running stock 7.1.2. The model is a conv net with inception, batch norm and dropout, trained using tensorflow slim.
Source code / logs
Commented out:
public float[] runInference(float[] pixels) {
        assertRightSize(pixels);
        final float[] outputArray = new float[128];
        // Simulate some sort of output
        Arrays.fill(outputArray, new Random().nextInt(1000)/new Random().nextFloat());
//     inferenceInterface.feed("phase_train", new bool[]{false});
//     inferenceInterface.feed("input", pixels, 1, 96, 96, 3);
//     inferenceInterface.run(new String[]{"output"});
        // Copy the output Tensor back into the output array.
//     inferenceInterface.fetch("output", outputArray);

        return outputArray;
    }

Enabled:
public float[] runInference(float[] pixels) {
        assertRightSize(pixels);
        final float[] outputArray = new float[128];
        inferenceInterface.feed("phase_train", new bool[]{false});
        inferenceInterface.feed("input", pixels, 1, 96, 96, 3);
        inferenceInterface.run(new String[]{"output"});
        // Copy the output Tensor back into the output array.
        inferenceInterface.fetch("output", outputArray);

        return outputArray;
    }

where float[] pixels is a float array of size 27648, denoting the pixels in an image of size 96x96.
The custom code is an update to the InferenceInterface to accept boolean types during feeding:
public void feed(String inputName, boolean[] src, long... dims) {
        byte[] b = new byte[src.length];
        for (int i = 0; i < src.length; ++i) {
            b[i] = (byte) (src[i] ? 1 : 0);
        }
        addFeed(inputName, Tensor.create(DataType.BOOL, dims, ByteBuffer.wrap(b)));
    }

Please let me know if there's any other information I can provide.

It looks to me like there's a related discussion about the overheads of feed/run/fetch in #8712. Does any of that help?

@cy89 thanks for the response. I read that issue before posting mine and it was related. Unfortunately, the discussion turned towards timing and performance as opposed to memory allocation.

Makes sense. @petewarden, are there perhaps any updates about our Android TF ecosystem and its memory usage behavior?

@faifai21 Thanks for the detailed report! Are you able to determine exactly how much the memory use increases per inference pass? If you change the size of your input array, does the rate of increase vary proportionally? I'm just curious if something else in the run() call could be allocating memory that never gets cleaned up.
@asimshankar Any idea why this might be happening, given that closeFetches() and closeFeeds() are both guaranteed to be called every invocation of TensorFlowInferenceInterface.run()?

There was a leak in 1.2.0-rc0 and prior versions,
fixed by https://github.com/tensorflow/tensorflow/commit/8304e197ea9eeb617f224a1ba0cc4068596098d1
@faifai21 : Since you're building from source, could you confirm that your build includes the commit mentioned above?

@asimshankar I just checked and that commit is included in the build. I can try rebuilding from the latest release and see if the issue goes away.

Are you able to determine exactly how much the memory use increases per inference pass?

@andrewharp I'll get you that information first thing tomorrow morning.

@asimshankar I pulled in the latest master and built against commit
https://github.com/tensorflow/tensorflow/commit/99a38ffd9d77c55ca6d0c373c6d4b72686284ac5. The memory leak is still occurring though.
@andrewharp It seems like memory is increasing by ~0.1 MB per inference pass, which makes sense as each input array is around 110kb. As well, increasing the size of the input array does cause the memory leak to increase:

With an input array of 96*96*3, each input array was 110kb. Memory usage was increasing at ~0.5 mb per sec. With 4.41 inputs processed per sec, it can be inferred that memory was increasing by ~0.11 mb per run.
With an input array of 120*120*3, each input array was 172kb. Memory usage was increasing at 0.7 mb per sec. Since input size was increased, we were processing slightly less inputs per sec than before. Assuming we were processing 4.1 inputs per sec now, we can infer that each run increased memory by ~0.17mb.

I was unable to increase input size by anymore, as that caused BufferOverflow exceptions with FloatBuffer as the input array was copied over when creating the Tensor.

I also tried decreasing the size of the input array to 30*30*3, which resulted in each input having a size of 10kb. The memory leak was still present, but it was much less apparent. As expected though, memory usage increased by ~0.01 mb per run.

@andrewharp We are also experiencing the same issue with tensorflow library on android. The memory keeps on increasing as the number of inference runs keep on increasing. This is leading to lot of out of memory errors in our application.
We tried creating a new TensorFlowInferenceInterface after every inference but even that did not solve the problem. It looks like a memory leak in tensorflow c++ library.

We tried creating a new TensorFlowInferenceInterface after every inference but even that did not solve the problem.

I tried the same thing, and can confirm that the memory leak was still there.

@andrewharp @asimshankar any updates on this?

We've also found this issue in our iOS tensorflow library (also built from source), which makes sense, given that the memory leak seems to be in C/C++ code.

Experiencing the same problem as above: lots of memory leakage due to the feeding operation.
While I'm sure I'm missing a lot of intricacies here, the source seems obvious to me. Each call to feed is allocating new memory for each tensor (float tensor in my case):
addFeed(inputName, Tensor.**create**(dims, FloatBuffer.wrap(src)));
Is there no way to overwrite the data in the existing FloatBuffer after the initial feed call? (instead of allocating space for an entirely new Tensor each time?) Would that not solve the memory leak we are experiencing?

@faifai21 : No updates yet unfortunately. Since you have this in C++ land as well, perhaps you could come up with a small C++ snippet that demonstrates this - and it might make it easier to trace any problems by running the sample on a desktop/laptop instead of a mobile device.
@Mr-Grieves : I'm not following. During every call to feed a new Tensor is created, but it is released with a call to closeFeeds() on every call to run().  Are you experiencing "leaks" or is it that total memory usage is higher than you'd want (and waits for a garbage collection cycle to come down)?

@asimshankar: The latter. I'm new to Java and still not totally comfortable with the idea of leaving objects around to be cleaned up by the GC... My app runs as desired initially, but slows down over time, and eventually grinds to a halt. I initially thought this issue was being caused by a memory leak, but perhaps the memory management is running as it should and the cause of the slowdown lies somewhere else...
I've attached a screenshot of my Memory and CPU monitors. GC events are being triggered every 10s or so, with 99% of the allocations occurring during the addFeed(inputName, Tensor.create(dims, FloatBuffer.wrap(src)));.
If the memory is not the problem, my issue is most likely CPU related, in which case I am posting in the wrong thread.

@Mr-Grieves It could be thermal throttling slowing down the CPU frequency after sustained load. I'm not sure if you can view this in Android Studio, but you could try printing the frequency manually via adb.

@asimshankar Thanks for the response. I've spoken to our iOS engineers a few more times and it seems like the memory leak does not actually happen in iOS land. This seems to be an issue on Android only, perhaps in the JNI methods?

@asimshankar @andrewharp This memory leak is blocking us from shipping tensorflow on Android. Are there any updates for this? Is there anything I can do to help? I tried looking through the source code to find the memory leak, but I'm not well versed in c/c++.

It has been 14 days with no activity and the awaiting tensorflower label was assigned. Please update the label and/or status accordingly.

Sorry this fell through the cracks. Is that still a problem with the latest versions?
Our current recommended platform for mobile is tflite, maybe that fits your needs better?

Nagging Awaiting Response: It has been 14 days with no activityand the awaiting response label was assigned. Is this still an issue?

Nagging Awaiting Response: It has been 14 days with no activityand the awaiting response label was assigned. Is this still an issue?

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!

@drpngx  I'm facing the exact same issue on version 1.5.0. As I keep performing inference on an Android device the memory consumed by the app keeps increasing.

No. of Inference runs
Memory usage

1
5.7 Mb

15
206 Mb

20
242 Mb

25
757 Mb

30
922 Mb

Is there a fix for this issue?

@deepaksuresh : There isn't any known fixes. Could you file a new issue with the details, particularly if there are any instructions to reproduce the problem? Can you share the model perhaps?

@asimshankar I'm running the model from a .pb file. It is 70 Mb, how can I share it with you?

authors: Mr-Grieves andrewharp asimshankar cy89 deepaksuresh drpngx faifai21 rhythm9229 tensorflowbutler

repo: tensorflow/tensorflow


text:
A very strange bug with tf.cond, update_ops and global_step

I'm using tf.Estimator with custom model_fn. When training, the estimator usually outputs log like:
INFO:tensorflow:loss = 1109.14, step = 1
INFO:tensorflow:loss = 937.876, step = 101 (6.245 sec)
INFO:tensorflow:loss = 632.192, step = 201 (6.195 sec)
By default, the printed steps should be 1, 101, 201... However, when I use the following function (which is simplified to reproduce the bug) in any place of the model:
def my_op(inputs, name=None):
  with tf.variable_scope(name, default_name='my_scope', reuse=False):
    count = tf.get_variable('count', shape=[],
      initializer=tf.zeros_initializer(), trainable=False)

    def myfunc1():
      return 1

    def myfunc2():
      tf.add_to_collection(tf.GraphKeys.UPDATE_OPS, count.assign_add(10.0))
      return 1

    tf.cond(tf.less(1, 2), myfunc1, myfunc2)
    return inputs

The log becomes something like:
INFO:tensorflow:loss = 1130.58, step = 0
INFO:tensorflow:loss = 940.298, step = 0 (6.352 sec)
The global step is always 0. After some tests, I found that the global step is not updated if myfunc2 is not executed. For example, if I write
    tf.cond(tf.less(count, 2), myfunc1, myfunc2)
then the global step is always 1.
I suspect that this is caused by the tf.add_to_collection(tf.GraphKeys.UPDATE_OPS, ...) in myfunc2, in which my intend is to update a variable when some condition holds. Maybe op created inside tf.cond can not be used as a dependency outside, but no error message is reported.
If I cannot add ops to UPDATE_OPS inside tf.cond, does this imply that stateful operations (like tf.layers.batch_normalization) can not be used inside tf.cond? So I cannot dynamically choose a network module from a set of network modules to execute if the network modules use any stateful operations like tf.layers.batch_normalization?
my tensorflow version: ('v1.4.0-rc0-10-g756a7fc', '1.4.0-rc1')

I have the same problem while using tf.train.Supervisor in TF nightly build 1118.

global_step was explicitly passed to the supervisor, but is not updated (so every summary collapsed to one point).
Models were saved in the same name.

My code was exactly the same as the one I was using in TF 1.4.0-rc0.

@skye might have some ideas behind the interaction between tf.cond and the global step.

It has been 14 days with no activity and the awaiting tensorflower label was assigned. Please update the label and/or status accordingly.

Nagging Assignee @skye: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @skye: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @skye: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @skye: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

The similar issue has appeared several times in #18795, #14809. I think it's a good feature to add an extra option to layers.batch_normalization, similar to that of contrib.batch_norm.

Nagging Assignee @skye: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @skye: It has been 30 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @skye: It has been 45 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @skye: It has been 60 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @skye: It has been 75 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @skye: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @skye: It has been 29 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @skye: It has been 44 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @skye: It has been 59 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @skye: It has been 74 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @skye: It has been 89 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @skye: It has been 104 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @skye: It has been 119 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @skye: It has been 134 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

This is a stale issue. Please check the issue with latest TensorFlow. If the issue still persists in the newer version of TF, please feel free to reopen it by providing details about the issue and a standalone code to reproduce the issue. Thanks!

I ran my test code again using the tensorflow/tensorflow:1.13.2-gpu-py3 docker image and got this result:
BEFORE ANYTHING IS RUN
0
[0. 0. 0.]
[[ 0.3860709   0.45507103 -0.1816141 ]
 [-0.00840878  0.7818753  -0.605694  ]
 [-0.04105538 -0.03619885 -0.10100174]
 [ 0.8570104   0.44614607  0.7425906 ]
 [ 0.7079007   0.8293366  -0.6455219 ]]
AFTER FIRST RUN:  we do not follow the path with the update op added
0
[0. 0. 0.]
[[ 0.3860709   0.45507103 -0.1816141 ]
 [-0.00840878  0.7818753  -0.605694  ]
 [-0.04105538 -0.03619885 -0.10100174]
 [ 0.8570104   0.44614607  0.7425906 ]
 [ 0.7079007   0.8293366  -0.6455219 ]]
AFTER SECOND RUN: we follow the path with the update op added
1
[0.00185758 0.00810567 0.02130217]
[[ 0.38625666  0.4558816  -0.17948389]
 [-0.00803727  0.78349644 -0.6014336 ]
 [-0.04049811 -0.03376715 -0.09461109]
 [ 0.85775346  0.44938833  0.75111145]
 [ 0.70882946  0.8333894  -0.63487077]]

This issue still exists in the latest version of Tensorflow. When the sess.run does not execute the tf.no_op, the opt.minimize operation silently fails to run.
I do not have authority to re-open this issue.

authors: JeremyCCHsu Multihuntr jvishnuvardhan ppwwyyxx tatatodd tensorflowbutler x10000year


text:
'No gradients provided for any variable, check your graph for ops that do not support gradients'

I write the code like following
import tensorflow as tf
input1=tf.Variable([1.0,2.0,3.0,4.0,5.0,6.0],name='input1')
input2=tf.Variable([2.0,3.0,4.0,6.0,8.0,9.0],name='input2')
values_range = tf.constant([0., 10.], dtype = tf.float32)
source_hist = tf.histogram_fixed_width(tf.to_float(input1), values_range, 11)
template_hist = tf.histogram_fixed_width(tf.to_float(input2), values_range, 11)
source_hist=tf.cast(source_hist,tf.float32)
template_hist=tf.cast(template_hist,tf.float32)
loss=2*tf.nn.l2_loss(source_hist-template_hist)
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    writer=tf.summary.FileWriter('./',sess.graph)
    train_step=tf.train.AdamOptimizer(0.001).minimize(loss)
    for i in range(0,10000,1):
        sess.run(train_step)
        print('input1_value',input1.eval())
        print('input2_value',input2.eval())
    writer.close()

Tensorflow throws an error and shows
''ValueError: No gradients provided for any variable, check your graph for ops that do not support gradients, between variables ["<tf.Variable 'input1:0' shape=(6,) dtype=float32_ref>", "<tf.Variable 'input2:0' shape=(6,) dtype=float32_ref>"] and loss Tensor("mul:0", shape=(), dtype=float32).''
I know the op  tf.histogram_fixed_width doesn't support backpropagation and is not differentiable. While
the op tf.floor has the same attribute as  tf.histogram_fixed_width. And the code below can run  without any error which surprises me a lot.
import tensorflow as tf
cst=tf.constant([1.2,1.4,2.8,4.6,6.8], dtype=tf.float32)
input=tf.Variable(cst)
new=tf.floor(input)
loss=2*tf.nn.l2_loss(input-new)
train_step=tf.train.AdamOptimizer(0.001).minimize(loss)
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for i in range(0,10000,1):
        sess.run(train_step)
        print('input_value',input.eval())
        print('new_value',new.eval())

I could not figure it out for days, please help

I have no idea what you are trying to accomplish.
Your loss is defined in terms of histogram_fixed_width, so your loss is not differentiable in terms of the variables, so you can't optimize this.  You seem to understand that based on "I know the tf.histogram_Fixed_width"...
Perhaps you want to try defining a gradient for histogram_fixed_width that is [None] like Floor has

Thanks so much for your reply. In fact my task is to implement the histogram loss mentioned in the paper Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses, which involves histogram match technique in the process of computing the loss function. According to the paper we can regard the data flows through the ops that are not differentiable as constants.
While I do not find the op Histogram_fixed_width in  'python/ops/math_grad.py' which mentioned in #897. So I add the function below directly in my code. The error disappears at last although it seems not elegant.
import tensorflow as tf
from tensorflow.python.framework import ops
@ops.RegisterGradient("Histogram_fixed_width")
def _Histogram_fixed_widthGrad(_, grad):
    return [None]

You can edit the source code and recompile tensorflow yourself, or else you can use the gradient override map
https://uoguelph-mlrg.github.io/tensorflow_gradients/
I would recommend asking this question in StackOverflow as others might have implemented this in tensorflow already.

Thanks for your advice！

@uptodiff were you able to find a solution? Currently facing this same issue.

@axelsly Did you find an answer? I'm also having the same issue.

authors: aselle axelsly mariusjohan uptodiff


text:
breaking change to distributed training moving from TF v1.3 to v1.4: “UnavailableError: Trying to connect an http1.x server”

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
Red Hat Enterprise Linux Server release 7.3 (Maipo)
TensorFlow installed from (source or binary):
source
TensorFlow version (use command below):
1.4 (moving from 1,3)
b'v1.4.0-4-g9283868' 1.4.0 in specific
Python version:
Python 3.6.2 :: Anaconda custom (64-bit)
Bazel version (if compiling from source):
0.5.4
GCC/Compiler version (if compiling from source):
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-11)
CUDA/cuDNN version:
N/a
GPU model and memory:
N/a
Exact command to reproduce:
see below. Appears to be any distributed training use case with the cluster spec defined as below.

Describe the problem
I believe this could be fairly characterized as a BUG.
I raised (and solved) this issue on stackoverflow here
In short, upgrading from tf v1.3 to 1.4 distributed training broke with the error
"tensorflow.python.framework.errors_impl.UnavailableError: Trying to connect an http1.x server".
I was training across multiple CPU cores all on the same localhost.
The fix was to change the ps to the  string "localhost" explicitly in the cluster spec instead of the IP "127.0.0.1". It stops trying to connect to the localhost via my proxy server (which indeed, was HTTP1.x only i think). I would call this a bug since that IP should be the equivalent, and I'm not sure what other kind of behavior might have inadvertently changed between versions (since this always worked in TF <= 1.3).
To be fair, I'm also not sure if this is a tensorflow or more gRPC specific issue. In either case, the above is a work around for the problem.

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code

Nagging Awaiting Response: It has been 14 days with no activityand the awaiting response label was assigned. Is this still an issue?

Nagging Awaiting Response: It has been 14 days with no activityand the awaiting response label was assigned. Is this still an issue?

It has been 14 days with no activity and the awaiting response label was assigned. Is this still an issue?

It has been 14 days with no activity and the awaiting response label was assigned. Is this still an issue?

It has been 29 days with no activity and the awaiting response label was assigned. Is this still an issue?

It has been 14 days with no activity and the awaiting response label was assigned. Is this still an issue?

It has been 15 days with no activity and the awaiting response label was assigned. Is this still an issue?

We are closing this issue for now due to lack of activity. Please comment if this is still an issue for you. Thanks!

authors: AngusSinclair tensorflowbutler