text:
1st Time Model taking too long to start

Please make sure that this is a bug. As per our
GitHub Policy,
we only address code/doc bugs, performance issues, feature requests and
build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary): pip install tensorflow==2.2
TensorFlow version (use command below):2.2
Python version:3.7
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: cuda 10.1, cudnn 7.6.5
GPU model and memory: NVIDIA MX110 2gb

You can collect some of this information using our environment capture
script
You can also obtain the TensorFlow version with:

TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"

TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"

v2.2.0-rc4-8-g2b96f3662b 2.2.0

Describe the current behavior
I Installed TF 2 and make a Sequential Model, That jupyter cell is taking forever to load.
Describe the expected behavior
It should load that cell fastly
Standalone code to reproduce the issue
from sklearn.datasets import load_iris
iris = load_iris()
X = iris.data
y = iris.target
from tensorflow.keras.models import Sequential
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.layers import Dense
y = to_categorical(y)
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X,y)
model1 = Sequential([
    Dense(512, activation='tanh', input_shape = X_train[0].shape),
    Dense(512//2, activation='tanh'),
    Dense(512//4, activation='tanh'),
    Dense(512//8, activation='tanh'),
    Dense(32, activation='relu'),
    Dense(3, activation='softmax')
])

Provide a reproducible test case that is the bare minimum necessary to generate
the problem. If possible, please share a link to Colab/Jupyter/any notebook.
Other info / logs Include any logs or source code that would be helpful to
diagnose the problem. If including tracebacks, please include the full
traceback. Large logs and files should be attached.
What I can see on my COmmand Line is
[I 16:37:13.657 NotebookApp] Starting buffering for 427db0cf-ee0b-49fa-b075-ce24cdc456bb:6564a2664692489a8473d40138e004c7
2020-08-22 16:38:17.412089: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-08-22 16:38:17.465067: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-08-22 16:38:17.465381: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce MX110 computeCapability: 5.0
coreClock: 1.006GHz coreCount: 2 deviceMemorySize: 1.96GiB deviceMemoryBandwidth: 37.33GiB/s
2020-08-22 16:38:17.465580: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-08-22 16:38:17.467132: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-08-22 16:38:17.468592: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-08-22 16:38:17.468928: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-08-22 16:38:17.470530: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-08-22 16:38:17.471467: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-08-22 16:38:17.474846: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-08-22 16:38:17.475036: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-08-22 16:38:17.475556: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-08-22 16:38:17.475947: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
2020-08-22 16:38:17.476205: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-08-22 16:38:17.482259: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 1800000000 Hz
2020-08-22 16:38:17.482562: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fa670000b20 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-08-22 16:38:17.482589: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-08-22 16:38:17.510525: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-08-22 16:38:17.510905: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55b17c39ce70 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-08-22 16:38:17.510933: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GeForce MX110, Compute Capability 5.0
2020-08-22 16:38:17.511104: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-08-22 16:38:17.511357: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce MX110 computeCapability: 5.0
coreClock: 1.006GHz coreCount: 2 deviceMemorySize: 1.96GiB deviceMemoryBandwidth: 37.33GiB/s
2020-08-22 16:38:17.511403: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-08-22 16:38:17.511423: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-08-22 16:38:17.511441: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-08-22 16:38:17.511458: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-08-22 16:38:17.511475: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-08-22 16:38:17.511493: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-08-22 16:38:17.511512: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-08-22 16:38:17.511562: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-08-22 16:38:17.511828: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-08-22 16:38:17.512056: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
2020-08-22 16:38:17.512092: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-08-22 16:38:17.513132: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-08-22 16:38:17.513144: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108]      0
2020-08-22 16:38:17.513150: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0:   N
2020-08-22 16:38:17.513236: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-08-22 16:38:17.513508: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-08-22 16:38:17.513756: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1461 MB memory) -> physical GPU (device: 0, name: GeForce MX110, pci bus id: 0000:01:00.0, compute capability: 5.0)
[I 16:39:14.659 NotebookApp] Saving file at /Desktop/Ahmad/REgular/iristest.ipynb

It is more than 10 mins, while I am waiting for that jupyter cell to load.

Hi @ahmadmustafaanis, can you provide a little more detail on the issue you're facing? I ran the code you provided in a colab notebook and didn't face any issues. The cell executed quickly.
Also please note that you are loading the data from sklearn and not Tensorflow, so the issue you're facing might be related to sklearn. If you comment out the model definition, does the cell still take a long time to execute?

Hi, @nikitamaia, yes only the particular cell takes a long time. This issue happens every time I run a new instance of a notebook.
After 5 or 6 mins, I am then able to train the models without any error. This long also takes when I run this cell,
model = Sequential([
Dense(10, input_shape=(19,)),
Dense(1)
])

When I jumped back to Tensorflow 2.0.0, It was working fine there. But when I am working with 2.3.0, or 2.2.0, it was taking a long time.
For this comment, I reinstalled 2.3.0, now I am getting this error message
---------------------------------------------------------------------------
InternalError                             Traceback (most recent call last)
<ipython-input-6-b597df38e227> in <module>
      1 model = tf.keras.models.Sequential([
      2     tf.keras.layers.Dense(10, input_shape=(10,)),
----> 3     tf.keras.layers.Dense(1)
      4 ])

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/training/tracking/base.py in _method_wrapper(self, *args, **kwargs)
    455     self._self_setattr_tracking = False  # pylint: disable=protected-access
    456     try:
--> 457       result = method(self, *args, **kwargs)
    458     finally:
    459       self._self_setattr_tracking = previous_value  # pylint: disable=protected-access

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/keras/engine/sequential.py in __init__(self, layers, name)
    115     # Skip the init in FunctionalModel since model doesn't have input/output yet
    116     super(functional.Functional, self).__init__(  # pylint: disable=bad-super-call
--> 117         name=name, autocast=False)
    118     self.supports_masking = True
    119     self._compute_output_and_mask_jointly = True

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/training/tracking/base.py in _method_wrapper(self, *args, **kwargs)
    455     self._self_setattr_tracking = False  # pylint: disable=protected-access
    456     try:
--> 457       result = method(self, *args, **kwargs)
    458     finally:
    459       self._self_setattr_tracking = previous_value  # pylint: disable=protected-access

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in __init__(self, *args, **kwargs)
    306     self._steps_per_execution = None
    307
--> 308     self._init_batch_counters()
    309     self._base_model_initialized = True
    310     _keras_api_gauge.get_cell('model').set(True)

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/training/tracking/base.py in _method_wrapper(self, *args, **kwargs)
    455     self._self_setattr_tracking = False  # pylint: disable=protected-access
    456     try:
--> 457       result = method(self, *args, **kwargs)
    458     finally:
    459       self._self_setattr_tracking = previous_value  # pylint: disable=protected-access

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in _init_batch_counters(self)
    315     # `evaluate`, and `predict`.
    316     agg = variables.VariableAggregationV2.ONLY_FIRST_REPLICA
--> 317     self._train_counter = variables.Variable(0, dtype='int64', aggregation=agg)
    318     self._test_counter = variables.Variable(0, dtype='int64', aggregation=agg)
    319     self._predict_counter = variables.Variable(

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/ops/variables.py in __call__(cls, *args, **kwargs)
    260       return cls._variable_v1_call(*args, **kwargs)
    261     elif cls is Variable:
--> 262       return cls._variable_v2_call(*args, **kwargs)
    263     else:
    264       return super(VariableMetaclass, cls).__call__(*args, **kwargs)

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/ops/variables.py in _variable_v2_call(cls, initial_value, trainable, validate_shape, caching_device, name, variable_def, dtype, import_scope, constraint, synchronization, aggregation, shape)
    254         synchronization=synchronization,
    255         aggregation=aggregation,
--> 256         shape=shape)
    257
    258   def __call__(cls, *args, **kwargs):

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/ops/variables.py in <lambda>(**kws)
    235                         shape=None):
    236     """Call on Variable class. Useful to force the signature."""
--> 237     previous_getter = lambda **kws: default_variable_creator_v2(None, **kws)
    238     for _, getter in ops.get_default_graph()._variable_creator_stack:  # pylint: disable=protected-access
    239       previous_getter = _make_getter(getter, previous_getter)

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/ops/variable_scope.py in default_variable_creator_v2(next_creator, **kwargs)
   2644       synchronization=synchronization,
   2645       aggregation=aggregation,
-> 2646       shape=shape)
   2647
   2648

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/ops/variables.py in __call__(cls, *args, **kwargs)
    262       return cls._variable_v2_call(*args, **kwargs)
    263     else:
--> 264       return super(VariableMetaclass, cls).__call__(*args, **kwargs)
    265
    266

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/ops/resource_variable_ops.py in __init__(self, initial_value, trainable, collections, validate_shape, caching_device, name, dtype, variable_def, import_scope, constraint, distribute_strategy, synchronization, aggregation, shape)
   1516           aggregation=aggregation,
   1517           shape=shape,
-> 1518           distribute_strategy=distribute_strategy)
   1519
   1520   def _init_from_args(self,

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/ops/resource_variable_ops.py in _init_from_args(self, initial_value, trainable, collections, caching_device, name, dtype, constraint, synchronization, aggregation, distribute_strategy, shape)
   1650             initial_value = ops.convert_to_tensor(
   1651                 initial_value() if init_from_fn else initial_value,
-> 1652                 name="initial_value", dtype=dtype)
   1653           if shape is not None:
   1654             if not initial_value.shape.is_compatible_with(shape):

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/framework/ops.py in convert_to_tensor(value, dtype, name, as_ref, preferred_dtype, dtype_hint, ctx, accepted_result_types)
   1497
   1498     if ret is None:
-> 1499       ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
   1500
   1501     if ret is NotImplemented:

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/framework/tensor_conversion_registry.py in _default_conversion_function(***failed resolving arguments***)
     50 def _default_conversion_function(value, dtype, name, as_ref):
     51   del as_ref  # Unused.
---> 52   return constant_op.constant(value, dtype, name=name)
     53
     54

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py in constant(value, dtype, shape, name)
    262   """
    263   return _constant_impl(value, dtype, shape, name, verify_shape=False,
--> 264                         allow_broadcast=True)
    265
    266

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py in _constant_impl(value, dtype, shape, name, verify_shape, allow_broadcast)
    273       with trace.Trace("tf.constant"):
    274         return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
--> 275     return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
    276
    277   g = ops.get_default_graph()

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py in _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
    298 def _constant_eager_impl(ctx, value, dtype, shape, verify_shape):
    299   """Implementation of eager constant."""
--> 300   t = convert_to_eager_tensor(value, ctx, dtype)
    301   if shape is None:
    302     return t

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/framework/constant_op.py in convert_to_eager_tensor(value, ctx, dtype)
     95     except AttributeError:
     96       dtype = dtypes.as_dtype(dtype).as_datatype_enum
---> 97   ctx.ensure_initialized()
     98   return ops.EagerTensor(value, ctx.device_name, dtype)
     99

~/anaconda3/envs/myenv/lib/python3.7/site-packages/tensorflow/python/eager/context.py in ensure_initialized(self)
    537         if self._use_tfrt is not None:
    538           pywrap_tfe.TFE_ContextOptionsSetTfrt(opts, self._use_tfrt)
--> 539         context_handle = pywrap_tfe.TFE_NewContext(opts)
    540       finally:
    541         pywrap_tfe.TFE_DeleteContextOptions(opts)

InternalError: CUDA runtime implicit initialization on GPU:0 failed. Status: device kernel image is invalid

Though I can see in command prompt that GPU is working, i.e when I executed the cell, this was output in terminal
2020-08-25 11:59:25.118216: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-08-25 11:59:25.119158: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce MX110 computeCapability: 5.0
coreClock: 1.006GHz coreCount: 2 deviceMemorySize: 1.96GiB deviceMemoryBandwidth: 37.33GiB/s
2020-08-25 11:59:25.119254: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-08-25 11:59:25.119313: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2020-08-25 11:59:25.119366: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2020-08-25 11:59:25.119422: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2020-08-25 11:59:25.119484: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2020-08-25 11:59:25.119545: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10
2020-08-25 11:59:25.119606: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2020-08-25 11:59:25.119779: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-08-25 11:59:25.120719: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-08-25 11:59:25.121499: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0

Anyways, This issue is not happening in Tf2.0.0 but in latest version. Thanks.

authors: ahmadmustafaanis nikitamaia


text:
Support tf.IsFinite lowering

1. System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Colab
TensorFlow installation (pip package or built from source): pip
TensorFlow library (version, if pip package or github SHA, if built from source): nightly

2. Code
import tensorflow as tf

@tf.function
def func(x):
  return tf.is_finite(x)

converter = tf.lite.TFLiteConverter.from_concrete_functions([func.get_concrete_function(tf.TensorSpec(shape=(1, 2, 3)))])
converter.convert()

3. Failure after conversion
It will fail if flex is not enabled. However, tf.is_inf() is supported without flex because it is lowered to other ops in lower_tf.cc. It should be easy to support lowering pattern of tf.is_finite. Note that tf.is_finite is also being used in tf.keras.layers.Softmax with multi-axes reduction. I understand that I can enable flex to use it, but it should be a minimum work to lower it directly.
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/keras/layers/advanced_activations.py#L328-L345

Thanks for filing the feature request! Is there any plan for this in your side?

@WindQAQ I'll make a quick PR to support op lowering.

Thanks for filing the feature request! Is there any plan for this in your side?

@abattery Is probably like tf.logical_not(tf.logical_or(tf.is_nan(x), tf.is_infinite(x)))

@WindQAQ I'll make a quick PR to support op lowering.

Thank you for the help!

Finally my PR is merged at e89316dfca814d2d2ff2bef47d33bc33140cc112
The tf.IsFinite is lowered to tf.Equal(tf.Subtract(x, x), 0).

source: gharchive/issue

authors: WindQAQ abattery thaink

repo: tensorflow/tensorflow

license_type: permissive

license_source: github-api


text:
No quantized winograd convolution in tflite

Winograd accelerate convolution both in float and int computation, I'm curious why there is no winograd convolution op in TFLite.  Weight transform is needed before weight quantization to keep accuracy as much as possible and then a new winograd_conv op is needed to handle the quantized transformed weight.
Is there any concern about adding a winograd conv op?

@multiverstack-intellif Sorry for the late response!
In order to expedite the trouble-shooting process here, could you please fill the issue template,
Thanks!

Closing as a stale. Please feel free to reopen if you still have any concern.
Thank you!

authors: multiverstack-intellif sushreebarsa


text:
NonMatchingChecksumError while downloading Caltech Birds 2011 dataset

Issue type
Bug
Have you reproduced the bug with TensorFlow Nightly?
Yes
Source
binary
TensorFlow version
tf 2.15
Custom code
Yes
OS platform and distribution
Google Colab
Mobile device
No response
Python version
3.10
Bazel version
No response
GCC/compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current behavior?
I am encountering a Non-Matching Checksum error while attempting to download the caltech_birds2011 dataset.
The error persists even when using the tfds-nightly package.
Standalone code to reproduce the issue
Code to reproduce the error:
import tensorflow_datasets as tfds
dataset = tfds.load('caltech_birds2011', split='train')

Colab:https://colab.research.google.com/drive/1jrI02VQ7ETw0oEL5RRk4vD-AQH07ZtIw?usp=sharing

Relevant log output
Downloading and preparing dataset 1.11 GiB (download: 1.11 GiB, generated: 1.11 GiB, total: 2.22 GiB) to /root/tensorflow_datasets/caltech_birds2011/0.1.1...
Dl Completed...: 100%
1/1 [00:00<00:00, 1.21 url/s]
Dl Size...:
0/0 [00:00<?, ? MiB/s]
---------------------------------------------------------------------------
NonMatchingChecksumError                  Traceback (most recent call last)
<ipython-input-2-b0dbf98bec1b> in <cell line: 3>()
      1 import tensorflow_datasets as tfds
      2 print("Version",tfds.__version__)
----> 3 dataset = tfds.load('caltech_birds2011', split='train')

19 frames
/usr/local/lib/python3.10/dist-packages/tensorflow_datasets/core/download/download_manager.py in _validate_checksums(url, path, computed_url_info, expected_url_info, force_checksums_validation)
    807         'https://www.tensorflow.org/datasets/overview#fixing_nonmatchingchecksumerror'
    808     )
--> 809     raise NonMatchingChecksumError(msg)
    810
    811

NonMatchingChecksumError: Artifact https://drive.google.com/uc?export=download&id=1hbzc_P1FuxMkcabkgn9ZKinBwW683j45, downloaded to /root/tensorflow_datasets/downloads/ucexport_download_id_1hbzc_P1FuxMkcabkgn9ZKinBw1sbZS_7v82APqMiPk0wej8WzZ5MmtVy61NUy-F6708.tmp.2f1a547e76e047aa9c2c734021a5bd45/download, has wrong checksum:
* Expected: UrlInfo(size=1.07 GiB, checksum='0c685df5597a8b24909f6a7c9db6d11e008733779a671760afef78feb49bf081', filename='CUB_200_2011.tgz')
* Got: UrlInfo(size=2.37 KiB, checksum='90974fd9547262b60bca7e22993f85c62ea79200cfee1bf4bef26e0334a0a7fc', filename='download')

@aditya02shah,
I was able to reproduce the issue on tensorflow v2.15. Could you please allow the developer team to deep dive into the issue and resolve the error. Thank you!

@tilakrayal Thank your for confirming and looking into the issue! Let me know if you need more details.

@aditya02shah,
Apologies for the delay. As per the documentation, NonMatchingChecksumError is raised due to the reasons below.
The website may be down, original datasets files may have been updated or Drive sometimes rejects downloads when too many people access the same URL.
As the workaround you can try to download from the link and also request to check in datasets repository for the quick resolution.
https://github.com/tensorflow/datasets/blob/master/tensorflow_datasets/image_classification/caltech_birds.py#L227
https://github.com/tensorflow/datasets/issues/1482
Thank you!

@tilakrayal Thanks for the assistance!


text:
Segmentation Fault in Shape Function When Accessing Attr

Adding this to shape function causes segfault (full example below):
      string data_format;
      Status s = c->GetAttr("test_str", &data_format);

On OS X:
$ bazel version
Build label: 0.4.3-homebrew
Build target: bazel-out/local-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Thu Dec 22 15:20:22 2016 (1482420022)
Build timestamp: 1482420022
Build timestamp as int: 1482420022

$git rev-parse
07bb8ea2379bd459832b23951fb20ec47f3fdbd4

Using the ZeroOut op but modifying the shape function:
#include "tensorflow/core/framework/op.h"
#include "tensorflow/core/framework/op_kernel.h"
#include "tensorflow/core/framework/shape_inference.h"

using namespace tensorflow;

REGISTER_OP("ZeroOut")
    .Input("to_zero: int32")
    .Output("zeroed: int32")
    .Attr("test_str: {'alex', 'bob'} = 'alex'")
    .SetShapeFn([](::tensorflow::shape_inference::InferenceContext* c) {
      string data_format;
      Status s = c->GetAttr("test_str", &data_format);
      c->set_output(0, c->input(0));
      return Status::OK();
    });

class ZeroOutOp : public OpKernel {
 public:
  explicit ZeroOutOp(OpKernelConstruction* context) : OpKernel(context) {}

  void Compute(OpKernelContext* context) override {
    // Grab the input tensor
    const Tensor& input_tensor = context->input(0);

    // Create an output tensor
    Tensor* output_tensor = NULL;
    OP_REQUIRES_OK(context, context->allocate_output(0, input_tensor.shape(),
                                                     &output_tensor));

  }
};

REGISTER_KERNEL_BUILDER(Name("ZeroOut").Device(DEVICE_CPU), ZeroOutOp);

and then running (within tf.TestCase):
class Alex(tf.test.TestCase):
    def test_alex(self):
        so_path = (cur_dir + "/../../../../" +
            "bazel-bin/tensorflow/core/user_ops/zero.so")
        my_module = tf.load_op_library(so_path)
        with self.test_session():
            print my_module.zero_out([1,1,2,1]).eval()

The segfault it self seems to be coming from protobuf. I filed a ticket there as well: https://github.com/google/protobuf/issues/2863.

I was unable to reproduce this on 1.0. I didn't run in a test session:
import tensorflow as tf
zero_out_module = tf.load_op_library('zero_out.so')
with tf.Session(''):
  zero_out_module.zero_out([[1, 1, 2, 4]]).eval()

I also ran with gcc instead of bazel.
I don't have any super compelling theories, but maybe try running in a regular session like above, using the g++ command given in the tutorial, and/or upgrading bazel?

I upgraded bazel and am still seeing the segfault:
$ bazel version
Build label: 0.4.5-homebrew
Build target: bazel-out/local-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Thu Mar 16 13:39:33 2017 (1489671573)
Build timestamp: 1489671573
Build timestamp as int: 1489671573

What is best way to compile with GCC?

Cool, thanks for narrowing that down! I'll work on reproing the bad bazel artifact, and I'm glad you found a workaround for now.

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!

wait... the last comment on this thread was @skye taking a look at the issue.

authors: cancan101 girving skye


text:
Update the directory of cuDNN in doc of install cuda

The directory of cuDNN has been changed to cuda. It would be better to update the doc.

Can one of the admins verify this patch?

I signed it!
On Thu, Jun 30, 2016 at 12:05 PM, googlebot notifications@github.com
wrote:

Thanks for your pull request. It looks like this may be your first
contribution to a Google open source project. Before we can look at your
pull request, you'll need to sign a Contributor License Agreement (CLA).
📝 Please visit https://cla.developers.google.com/
https://cla.developers.google.com/ to sign.
Once you've signed, please reply here (e.g. I signed it!) and we'll
verify. Thanks.

If you've already signed a CLA, it's possible we don't have your
GitHub username or you're using a different email address. Check your
existing CLA data https://cla.developers.google.com/clas and verify
that your email is set on your git commits
https://help.github.com/articles/setting-your-email-in-git/.
If you signed the CLA as a corporation, please let us know the
company's name.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/pull/3118#issuecomment-229554346,
or mute the thread
https://github.com/notifications/unsubscribe/AClteCvWyIz76lCQFx0ug3b_pzZBw3X2ks5qQ0CdgaJpZM4JByDM
.

Thanks!

Thanks for merging this @martinwicke 😃

source: gharchive/pull-request


text:
TFLu: Fix Ethos-U build issue

The issue is that a binary need to be built twice, since it
depends on recursive_find. The driver is not fully downloaded when
recursive_find is called.
The solution proposal is to call the download script
immediately via the make's shell function.
Also add missing exit to error case in download script.

Thanks for contributing to TensorFlow Lite Micro.
To keep this process moving along, we'd like to make sure that you have completed the items on this list:
Read the contributing guidelines for TensorFlow Lite Micro
Created a TF Lite Micro Github issue
Linked to the issue from the PR description
We would like to have a discussion on the Github issue first to determine the best path forward, and then proceed to the PR review.

I'll let @petewarden review this since he is most familiar with the Makefile logic and its repercussions.

Looks good from my side.

@petewarden Ping for review

authors: advaitjain freddan80 gbaned mansnils


text:
[TFTRT] Additional ConvertPool validation

Add additional validation for TRT failure cases for ConvertPool and ConvertPool3D, so that failing nodes will not be converted.

Hi @nvkevihu Can you please check @bixia1's comments and keep us posted ? Thank you!

Thanks; squashed!

authors: gbaned nvkevihu


text:
Add MNIST LSTM model training

Train/save/convert LSTM model on MNIST dataset
Using python3:
python3 tensorflow/lite/micro/examples/mnist_lstm/train.py
Using bazel:
bazel build tensorflow/lite/micro/examples/mnist_lstm:train
bazel-bin/tensorflow/lite/micro/examples/mnist_lstm/train
Terminal arguments:

--epoch : number of epochs for model training, default 1 for fast trainning
--save_dir: the directory to save the trained model, default to /tmp/trained_model.
--save_tf_model: if specified, save the intermediate tensorflow model (debug usage)
--optimize_conversion: if specified, convert tflite model using dynamic range quantization

Example:
bazel build tensorflow/lite/micro/examples/mnist_lstm:train
bazel-bin/tensorflow/lite/micro/examples/mnist_lstm/train --epoch=10 --save_dir=trained_model --optimize_conversion
Code above will train a simple LSTM model for MNIST digits recognition with 10 epochs. The trained model is converted and  quantized to tflite flatbuffer format, and the .tflite file is saved to the trained_model folder in the source directory. Note the intermediate tensorflow model is not saved in this case (pass --save_tf_model to enable it).

ready for review again, PTAL

repo: tensorflow/tflite-micro

license_source: bigquery


text:
Docker Installation

Where can I find how to install, configure and use TensorTrade (docker version) for Windows?
Which version is the latest and official?
https://hub.docker.com/search?q=tensortrade&type=image

The dockerfile is provided within the master source repository. Please use this one not a hosted model from dockerhub.

authors: YuriyTigiev rhamnett

repo: tensortrade-org/tensortrade


text:
Command queue commands should be smaller

Right now, a command takes up 22KB. Once I support double-slotting commands, that's 44KB of space I am taking up just for commands, which is quite a bit.
The main reason commands are this big is because of unicasting runtime arguments to 108 worker cores (theoretical max). One possibility to lower the command size is to send multiple device commands in case we have lots of runtime arguments, however this should be more thoroughly thought out.
fyi @davorchap

implemented in FD2

authors: DrJessop davorchap

repo: tenstorrent-metal/tt-metal


text:
Incorrect front padding result with ttnn.pad when padding length < 16

When doing front padding with ttnn.pad in ROW_MAJOR layout, the padded result is incorrect when front padding length < 16
Reproduce
Run the example below:
import torch
import ttnn

def main(device):
    ptensor = torch.ones(32, dtype=torch.bfloat16)
    tensor = ttnn.from_torch(ptensor, layout=ttnn.ROW_MAJOR_LAYOUT, device=device)
    tensor = ttnn.pad(tensor, padding=[(2, 0)], value=0.5)
    print(tensor)

if __name__ == "__main__":
    device_id = 0
    device = ttnn.open_device(device_id=device_id)
    main(device)

Expected output:
ttnn.Tensor([ 0.50000,  0.50000,  ...,  1.00000,  1.00000], shape=Shape([34]), dtype=DataType::BFLOAT16, layout=Layout::ROW_MAJOR)

Actual output:
ttnn.Tensor([ 1.00000,  1.00000,  ...,  0.50000,  0.50000], shape=Shape([34]), dtype=DataType::BFLOAT16, layout=Layout::ROW_MAJOR)

However, if we change the the front padding length to 16 [(16, 0)], then it pads correctly
ttnn.Tensor([ 0.50000,  0.50000,  ...,  1.00000,  1.00000], shape=Shape([48]), dtype=DataType::BFLOAT16, layout=Layout::ROW_MAJOR)

These input varation are appeared in models and is related to this constraint
        ["Tensor<[1, 144, 56, 56]> self = ?", "List[int] pad = [1, 2, 1, 2]", "number value = 0.0"],
        ["Tensor<[1, 672, 14, 14]> self = ?", "List[int] pad = [1, 2, 1, 2]", "number value = 0.0"],
        ["Tensor<[1, 144, 60, 60]> self = ?", "List[int] pad = [1, 2, 1, 2]", "number value = 0.0"],
        ["Tensor<[1, 672, 15, 15]> self = ?", "List[int] pad = [2, 2, 2, 2]", "number value = 0.0"],
        ["Tensor<[1, 144, 65, 65]> self = ?", "List[int] pad = [2, 2, 2, 2]", "number value = 0.0"],
        ["Tensor<[1, 288, 33, 33]> self = ?", "List[int] pad = [1, 1, 1, 1]", "number value = 0.0"],
        ["Tensor<[1, 720, 17, 17]> self = ?", "List[int] pad = [2, 2, 2, 2]", "number value = 0.0"],
        ["Tensor<[1, 192, 75, 75]> self = ?", "List[int] pad = [2, 2, 2, 2]", "number value = 0.0"],
        ["Tensor<[1, 816, 19, 19]> self = ?", "List[int] pad = [2, 2, 2, 2]", "number value = 0.0"],
        ["Tensor<[1, 192, 95, 95]> self = ?", "List[int] pad = [2, 2, 2, 2]", "number value = 0.0"],
        ["Tensor<[1, 960, 24, 24]> self = ?", "List[int] pad = [1, 2, 1, 2]", "number value = 0.0"],


text:
Remove passed conv2d cases from guard

Ticket
Link to Github Issue
Problem description
Provide context for the problem
What's changed
Describe the approach used to solve the problem.
Summarize the changes made and their impact

Failed on n150


text:
generate_api_json.sh - sed in-place syntax differences between BSD and GNU

It looks like GNU (Linux) and BSD(MacOS) have incompatible ways of defining in-place file replacement for sed. This is the best explanation I could find:
https://stackoverflow.com/questions/34281077/how-to-make-sed-i-work-in-both-mac-and-ubuntu
I came up with two possible solutions:

Use perl instead of sed

perl -i -pe 'BEGIN{ $api = `cat api.json`; chomp $api } s/%REPLACEME%/\n$api/' bin/playwright_server

Backup bin/playwright_server to bin/playwright_server.bak and delete the file afterwards. This works on Linux and MacOS both:

sed -i.bak -e '/%REPLACEME%/r api.json' -e 's/%REPLACEME%//g' bin/playwright_server
rm bin/playwright_server.bak

Would one of these solutions be OK? I could also add some updates to quiet the program under normal execution (like using mkdir -p and git pull -q.

Yes, either would be fine.  The latter is probably an easier implementation -- send me a PR and I will merge it immediately.

authors: kcaran teodesian

repo: teodesian/playwright-perl

license: MIT


text:
odd state, gcode parser borked

I had a probe routine crash the GUI.  After a reset I was stuck with this state.  I was trying to send
g91 y-1

It would be handy to have a logging function so you can see what is sent, and what msgs grbl is sending.

I had set $10=3 to see the machine coordinates (that would be nice on the DRO pane), and I think I broke the reporting.  I did $RST=* and reset it but the GUI isn't getting any updates and is stuck in "waiting for controller" state.

somehow the config was connecting to COM1 which isn't connected to the board.  Not sure how that happened.

repo: terjeio/Grbl-GCode-Sender


text:
Authorizer implementation

Hi Anton, I like this module a lot! Do you have plans to add authorizer support to it? It's something I'm possibly interested in working on. Do you have ideas about how the implementation should look?

Hi Alex! Recently there was #17 about authorization on routes.
If you are thinking about the creation of aws_apigatewayv2_authorizer resources by this module, then I don't have immediate plans to add those because I don't have a need for them myself.
From the implementation point of view, aws_apigatewayv2_authorizer resources should be similar to the relation between var.integrations and aws_apigatewayv2_route resources.
I will be glad to review the implementation if anyone wants to work on it.

I've created a PR here:
https://github.com/terraform-aws-modules/terraform-aws-apigateway-v2/pull/64
@antonbabenko  do you mind giving a review please? :)


text:
docs: Update autoscaler installation in IRSA example

PR o'clock
Description
This PR fixes two issues in the irsa example:

Wrong chart values (or not compatible with the latest version). Please check this line: https://github.com/helm/charts/blob/a5632b118a97e829d9b93312ddc5816c3025d00b/stable/cluster-autoscaler/templates/serviceaccount.yaml#L8
Updates the helm install command used as the Helm chart has been moved to the kubernetes/autoscaler repository.

Checklist

[ ] CI tests are passing
[ ] README.md has been updated after any changes to variables and outputs. See https://github.com/terraform-aws-modules/terraform-aws-eks/#doc-generation

Thanks @aryklein for your contribution.

Thanks @aryklein for your contribution.

My pleasure!

authors: aryklein barryib

repo: terraform-aws-modules/terraform-aws-eks


text:
De-duplicate detection and omission

Is your request related to a new offering from AWS?
No
Is your request related to a problem? Please describe.
Yes, one direct use-case is when using GuardDuty on multiple regions account-level issues will all generate the same(ish) output and cause some insane spam on your Slack channel.
Describe the solution you'd like.
An option to enable/disable some intelligent duplicate detection within' this stack.  This could be globally, or per alert-type.  This could be rather simple or extremely complex.
Describe alternatives you've considered.
I suppose a "de-dupe" SNS forwarding topic/service could be used instead if such an thing exists.  A google search didn't find any such examples out there.
Additional context
See example:

Note: This feature depending how it is implemented may end up making this stack cost quite a bit more to host by having to have an ElasticCache deployed.  Alternatively, could use something more serverless-friendly like DynamoDB which would have minimal added cost and complexity to this stack.  Either way it adds to this stack's complexity.

un-stale

I don't think we will be adding such deduplication logic into the module because this module does just one thing - forward messages to Slack.
In order to achieve the deduplication logic, it has to happen before calling this module.
Something like, GuardDuty/CloudTrail/etc -> SNS topic -> SQS queue to implement deduplication (read more) -> SNS topic clean -> This module to deliver message to Slack.
So, I am going to close this issue.

authors: AndrewFarley antonbabenko


text:
Current db_parameter_group module doesn't work with Postgres

The parameters listed here in Line 24 are MySQL specific.

Fixed by #2

authors: antonbabenko khosrow

repo: terraform-aws-modules/terraform-aws-rds


text:
Adding conditional flag to enable read replica if required

HI team,
We are using terraform open-source terraform module for creating postgres sql instance...where we are using same code for creating postgres sql instance in multiple environments like dev, staging & production.
Where we require read replica instance only for production, it's not needed for dev & staging. With the current master code we are not able to restrict this approach, so I have added a count condition with a boolean flag. Please have a review on this PR and let me know if this make sense. Also let me know if i have missed something from my end.

Why do you need a conditional flag? You can already set read_replicas to an empty list.


text:
chore(deps): update common-dev-assets digest to 22ebbcb

This PR contains the following updates:

Package
Update
Change

common-dev-assets
digest
b3e214a -> 22ebbcb

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

/run pipeline

:tada: This PR is included in version 1.0.5 :tada:
The release is available on:

GitHub release
v1.0.5

Your semantic-release bot :package::rocket:

repo: terraform-ibm-modules/mock-module


text:
add name and desc

Description
adds name and description to the pipeline
Check the relevant boxes:

[ ] Bug fix (nonbreaking change that fixes an issue)
[ ] New feature (nonbreaking change that adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Examples or tests (addition or updates of examples or tests)
[ ] Documentation update
[x] CI related update (pipeline, etc.)

Checklist

[ ] If relevant, a test for the change has been added or updated as part of this PR.
[ ] If relevant, documentation for the change has been added or updated as part of this PR.

Merge

Merge using "Squash and merge".
Make sure to use a relevant conventional commit message that is based on the PR contents. The commit message determines whether a new version of the modules needs to be released, and if so, which semver number to use).

@SirSpidey Can you rebase with main branch please?

authors: SirSpidey ocofaigh

repo: terraform-ibm-modules/repo-settings


text:
chore(deps): update common-dev-assets digest to d50e7e6

This PR contains the following updates:

Package
Update
Change

common-dev-assets
digest
0e4338f -> d50e7e6

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

/run pipeline


text:
Add operating-system input variable for OCP 4.15 and beyond (RHCoreOS now an option)

Description
Add operating-system input variable as Red Hat CoreOS is now an option in OCP 4.15 and above.
New or affected modules
all modules that create a ROKS cluster

By submitting this issue, you agree to follow our Code of Conduct

PR merged, Closing this issue


text:
chore(deps): update ci dependencies

This PR contains the following updates:

Package
Type
Update
Change

common-dev-assets

digest
20ea4bb -> 2520c3d

github.com/terraform-ibm-modules/ibmcloud-terratest-wrapper
require
patch
v1.23.9 -> v1.23.10

Release Notes

terraform-ibm-modules/ibmcloud-terratest-wrapper (github.com/terraform-ibm-modules/ibmcloud-terratest-wrapper)
v1.23.10
Compare Source
Bug Fixes

deps: update module github.com/gruntwork-io/terratest to v0.46.1 (#682) (dec9602)

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

/run pipeline

:tada: This PR is included in version 1.2.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


text:
feat: add cross region support for buckets

Description
CRA scan is failing with Goal ID 3000116 : Check whether Cloud Object Storage bucket resiliency is set to cross region
For this to pass, I think we need to add cross region support to https://github.com/terraform-ibm-modules/terraform-ibm-cos
The ibm_cos_bucket terraform resource supports:
cross_region_location - (Optional, String) Specify the cross-regional bucket location. Supported values are us, eu, and ap. If you use this parameter, do not set single_site_location or region_location at the same time.
so we just need to expose this in the module.
Check the relevant boxes:

[ ] Bug fix (nonbreaking change that fixes an issue)
[x] New feature (nonbreaking change that adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Examples or tests (addition or updates of examples or tests)
[ ] Documentation update
[ ] CI related update (pipeline, etc.)

Checklist

[ ] If relevant, a test for the change has been added or updated as part of this PR.
[ ] If relevant, documentation for the change has been added or updated as part of this PR.

Merge

Merge using "Squash and merge".
Make sure to use a relevant conventional commit message that is based on the PR contents. The commit message determines whether a new version of the modules needs to be released, and if so, which semver number to use).

I think we should also create a new example that will show how to use cross region as apposed to a single region, and run it as part of pr_test.go
We should then be able to update the CRA scan to scan that example, and remove the 3000116 goal from the CRA ignore list json file

I think passing a property is insufficient. If you are lucky you might be able to force region_location to null (when setting cross_region_location). Also force cross_region_location to null when setting region_location.
Otherwise, like with key_protect; it may be necessary to create two more bucket definitions. Similar model, instead of 1 of 2 being used, it becomes 1 of 4 and the right id/name/public/private have to be set in the locals (L188) so that outputs work as expected.

I think we can pass null value for one or the other, but we will either need a new variable to say whether to do single region or cross region, or else some validation which fails if a value is passed for both

I think we can pass null value for one or the other, but we will either need a new variable to say whether to do single region or cross region, or else some validation which fails if a value is passed for both

I've added some validation to limit the number of values you can pass to cross_region_location. Also added some validation in the locals to verify both values region and cross_region_location cannot be set at the same time. I'm still quite new to terraform so maybe this isn't sufficient enough.

@jor2 don't commit updates to the common-dev-assets git submodule, or you will end up with conflicts very quickly (like you have now) due to the renovate automation we have running several times a day that bumps in in main branch. Suggest removing it from this PR

@jor2 don't commit updates to the common-dev-assets git submodule, or you will end up with conflicts very quickly (like you have now) due to the renovate automation we have running several times a day that bumps in in main branch. Suggest removing it from this PR

makes sense, those conflicts are very annoying 😠.

Hmm, the test failed with:
Error: failed to update the lifecyle rule on COS bucket cos-complete-2nsjgr-bucket-2, InvalidRequest: The operation requested is not supported
I dont know if this is another feature that is unsupported with cross region? Were you able to successfully run this locally?

@jor2 please try to avoid force push, should be used as a last resort. It can be unsafe because it rewrites the commit history. For more details see https://www.git-tower.com/blog/force-push-in-git/

:tada: This PR is included in version 5.3.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

authors: daniel-butler-irl jor2 ocofaigh shemau terraform-ibm-modules-ops

repo: terraform-ibm-modules/terraform-ibm-cos


text:
fix: add a count around the deprecation notice for var.service_endpoints so it only triggers if variable is used

Description
add a count around the deprecation notice for var.service_endpoints so it only triggers if variable is used
Types of changes in this PR
No release required

[ ] Examples or tests (addition or updates of examples or tests)
[ ] Documentation update
[ ] CI-related update (pipeline, etc.)
[ ] Other changes that don't affect Terraform code

Release required

[x] Bug fix (patch release (x.x.X): Change that fixes an issue and is compatible with earlier versions)
[ ] New feature (minor release (x.X.x): Change that adds functionality and is compatible with earlier versions)
[ ] Breaking change (major release (X.x.x): Change that is likely incompatible with previous versions)

Release notes content
Replace this text with information that users need to know about the bug fixes, features, and breaking changes. This information helps the merger write the commit message that is published in the release notes for the module.

Checklist for reviewers

[ ] The PR references a GitHub issue.
[ ] If relevant, a test for the change is included or updated with this PR.
[ ] If relevant, documentation for the change is included or updated with this PR.

Merge actions for mergers

Merge by using "Squash and merge".

Use a relevant conventional commit message that is based on the PR contents and any release notes provided by the PR author.
The commit message determines whether a new version of the module is needed, and if so, which semver increment to use (major, minor, or patch).

I think I just bumped into the issue this PR has on another branch.
I think you need to add this to resolve the issue:
diff --git a/examples/complete/main.tf b/examples/complete/main.tf
index cc26c9e..e513500 100644
--- a/examples/complete/main.tf
+++ b/examples/complete/main.tf
@@ -101,7 +101,6 @@ module "cbr_zone" {

 # Create COS instance and Key protect instance.
 # Create COS bucket-1 with:
-# - Retention
 # - Encryption
 # - Monitoring
 # - Activity Tracking
@@ -118,6 +117,7 @@ module "cos_bucket1" {
   key_protect_key_crn                = module.key_protect_all_inclusive.keys["${local.key_ring_name}.${local.key_name}"].crn
   sysdig_crn                         = module.observability_instances.sysdig_crn
   activity_tracker_crn               = local.at_crn
+  retention_enabled                  = false
   bucket_cbr_rules = [
     {
       description      = "sample rule for bucket 1"
@@ -178,7 +178,6 @@ module "cos_bucket1" {

 # We will reuse the COS instance, Key Protect instance and Key Protect Key Ring / Key that were created in cos_bucket1 module.
 # Create COS bucket-2 with:
-# - Retention
 # - Cross Region Location
 # - Encryption
 # - Monitoring
@@ -196,6 +195,7 @@ module "cos_bucket2" {
   create_cos_instance      = false
   existing_cos_instance_id = module.cos_bucket1.cos_instance_id
   key_protect_key_crn      = module.key_protect_all_inclusive.keys["${local.key_ring_name}.${local.key_name}"].crn
+  retention_enabled        = false
   bucket_cbr_rules = [
     {
       description      = "sample rule for bucket 2"

To avoid creating the buckets with retention, to enable the key to be deleted.
They will still be encrypted with the key, they will just be deletable and the dependancy tree should make sure the buckets are deleted before the key.

:tada: This PR is included in version 5.6.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


text:
chore(deps): update ci dependencies

This PR contains the following updates:

Package
Type
Update
Change

common-dev-assets

digest
1d683d9 -> f9433ef

github.com/terraform-ibm-modules/ibmcloud-terratest-wrapper
require
minor
v1.22.0 -> v1.23.0

Release Notes

terraform-ibm-modules/ibmcloud-terratest-wrapper (github.com/terraform-ibm-modules/ibmcloud-terratest-wrapper)
v1.23.0
Compare Source
Features

added json comparison functions (#593) (1781c81)

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

/run pipeline

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


text:
chore(deps): update dependency go to v1.22.3

This PR contains the following updates:

Package
Type
Update
Change

go (source)
toolchain
patch
1.22.2 -> 1.22.3

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

/run pipeline

/run pipeline

:tada: This PR is included in version 1.1.3 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


text:
chore(deps): update common-dev-assets digest to f9f1ac0

This PR contains the following updates:

Package
Update
Change

common-dev-assets
digest
cd97e7d -> f9f1ac0

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

Edited/Blocked Notification
Renovate will not automatically rebase this PR, because it does not recognize the last commit author and assumes somebody else may have edited the PR.
You can manually request rebase by checking the rebase/retry box above.
⚠ Warning: custom changes will be lost.

:tada: This PR is included in version 2.1.0 :tada:
The release is available on:

GitHub release
v2.1.0

Your semantic-release bot :package::rocket:


text:
chore(deps): update common-dev-assets digest to aca6ee1

This PR contains the following updates:

Package
Update
Change

common-dev-assets
digest
67d664c -> aca6ee1

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

:tada: This PR is included in version 2.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


text:
chore(deps): update common-dev-assets digest to 58a2af7

This PR contains the following updates:

Package
Update
Change

common-dev-assets
digest
05291e2 -> 58a2af7

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

/run pipeline

:tada: This PR is included in version 2.3.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


text:
chore(deps): update common-dev-assets digest to c92be0f

This PR contains the following updates:

Package
Update
Change

common-dev-assets
digest
30da825 -> c92be0f

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

/run pipeline

:tada: This PR is included in version 7.4.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


text:
chore(deps): update common-dev-assets digest to c990c99

This PR contains the following updates:

Package
Update
Change

common-dev-assets
digest
f9f1ac0 -> c990c99

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

:tada: This PR is included in version 2.6.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


text:
chore(deps): update common-dev-assets digest to 837976a

This PR contains the following updates:

Package
Update
Change

common-dev-assets
digest
eac64e3 -> 837976a

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, click this checkbox.

This PR has been generated by Renovate Bot.

:tada: This PR is included in version 2.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


text:
chore(deps): update module github.com/terraform-ibm-modules/ibmcloud-terratest-wrapper to v1.42.8

This PR contains the following updates:

Package
Type
Update
Change

github.com/terraform-ibm-modules/ibmcloud-terratest-wrapper
require
patch
v1.42.7 -> v1.42.8

Release Notes

terraform-ibm-modules/ibmcloud-terratest-wrapper (github.com/terraform-ibm-modules/ibmcloud-terratest-wrapper)
v1.42.8
Compare Source
Bug Fixes

deps: update module github.com/go-git/go-git/v5 to v5.13.0 (#908) (94d589b)

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

ℹ Artifact update notice
File name: tests/go.mod
In order to perform the update(s) described in the table above, Renovate ran the go get command, which resulted in the following additional change(s):

6 additional dependencies were updated

Details:

Package
Change

github.com/ProtonMail/go-crypto
v1.0.0 -> v1.1.3

github.com/cyphar/filepath-securejoin
v0.2.4 -> v0.2.5

github.com/go-git/go-billy/v5
v5.5.0 -> v5.6.0

github.com/go-git/go-git/v5
v5.12.0 -> v5.13.0

github.com/skeema/knownhosts
v1.2.2 -> v1.3.0

golang.org/x/net
v0.31.0 -> v0.33.0

/run pipeline


text:
chore(deps): update terraform-module

This PR contains the following updates:

Package
Type
Update
Change

terraform-ibm-modules/icd-postgresql/ibm (source)
module
patch
3.9.0 -> 3.9.1

terraform-ibm-modules/landing-zone-vpc/ibm (source)
module
minor
7.11.0 -> 7.13.1

terraform-ibm-modules/security-group/ibm (source)
module
minor
2.2.0 -> 2.3.0

Release Notes

terraform-ibm-modules/terraform-ibm-icd-postgresql (terraform-ibm-modules/icd-postgresql/ibm)
v3.9.1
Compare Source
Bug Fixes

updated the admin_pass variable description since admin password length requirement has change to a minimum of 15 characters (#331) (18b4392)

terraform-ibm-modules/terraform-ibm-landing-zone-vpc (terraform-ibm-modules/landing-zone-vpc/ibm)
v7.13.1
Compare Source
Bug Fixes

deps: update terraform terraform-ibm-modules/resource-group/ibm to v1.1.4 (#671) (f03b300)

v7.13.0
Compare Source
Features

cleanup, updated variables and basic example (#670) (b1a38d0)

v7.12.0
Compare Source
Features

added support to create resources in existing VPC and use existing subnets. See Existing VPC and subnets Example (#638) (f0c5b08)

terraform-ibm-modules/terraform-ibm-security-group (terraform-ibm-modules/security-group/ibm)
v2.3.0
Compare Source
Features

add an example for manipulating kube sgs (#197) (fa97d48)

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

:tada: This PR is included in version 4.1.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


text:
Removing usages of get/setMasterCs in getDpaXs

Description
As part of #930, this PR removes all 3 usages of getMasterCs() and setMasterCs() from Block.getDpaXs().  This method is too specific to be on Block anyway, but if you move it to the GlobalFluxInterface (the only place it is every called), it becomes much simpler.

Checklist

[X] This PR has only one purpose or idea.
[X] Tests have been added/updated to verify that the new/changed code works.

[X] The code style follows good practices.
[X] The commit message(s) follow good practices.

[X] The release notes (location doc/release/0.X.rst) are up-to-date with any bug fixes or new features.
[X] The documentation is still up-to-date in the doc folder.
[X] The dependencies are still up-to-date in setup.py.

I'm good on this, if you wanted to merge.

authors: john-science keckler

repo: terrapower/armi


text:
自定义消息机制

参考
messageBridge.zip

使用插件系统统一处理

repo: terwer/src-sy-post-publisher


text:
Gafana errors after importing teslamate postgres database

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
SOLVED!  - SEE HERE: https://github.com/teslamate-org/teslamate/issues/3976#issuecomment-2181302937
Failed attempt to "move" (backup and restore) a full teslamate database between two different hosts (servers)!

The Source host was using CentOS 7 and an older postgres:12 database and was configured for separate FQDN's:
teslamate.example.com & garfana.example.com

The Target host was setup with Ubuntu Server 22.04 and the newest postgres:16 database, as well as one FQDN with a subpath:
tm.example.com & tm.example.com/grafana

Both hosts confirmed running latest Docker (v26.0.0 / v26.1.4) and Docker Compose (vv2.25.0 / v2.27.1) versions.

These are the steps taken with great care and with no errors occurred in the import process, only in Grafana when accessing the imported data:
A - Ensured teslamate on Source host was running properly and was recording data from the Tesla API.
B - Upgraded Source host teslamate from v1.29.1 to latest current version of v1.29.2
C - Exported Source host teslamate postgres:12 database into a file without any errors using:
docker compose exec -T database pg_dump -U teslamate teslamate > teslamate.bck
D - Deployed a fresh teslamate v1.29.2 environment on the Target host, using postgres:16 and single FQDN for Grafana as tm.example.com/grafana
E - Prepared Target host for database import, as follows:
E1 - Stopped the new teslamate container
E2 - Accessed new postgres:16 DB and issued cleanup commands as per HERE (with no errors) using:
docker exec -i teslamate-postgres /bin/bash -c "PGPASSWORD=<TM_DB_PASS> psql --username teslamate teslamate" << .
Result was:
> drop schema public cascade;
> create schema public;
> create extension cube;
> create extension earthdistance;
> CREATE OR REPLACE FUNCTION public.ll_to_earth(float8, float8)
> RETURNS public.earth
> LANGUAGE SQL
> IMMUTABLE STRICT
> PARALLEL SAFE
> AS 'SELECT public.cube(public.cube(public.cube(public.earth()*cos(radians(\$1))*cos(radians(\$2))),public.earth()*cos(radians(\$1))*sin(radians(\$2))),public.earth()*sin(radians(\$1)))::public.earth';
> .
NOTICE:  drop cascades to 25 other objects
DETAIL:  drop cascades to extension cube
drop cascades to extension earthdistance
drop cascades to type billing_type
drop cascades to type range
drop cascades to type states_status
drop cascades to type unit_of_length
drop cascades to type unit_of_pressure
drop cascades to type unit_of_temperature
drop cascades to function convert_celsius(numeric,text)
drop cascades to function convert_km(numeric,text)
drop cascades to function convert_m(double precision,text)
drop cascades to function convert_tire_pressure(numeric,character varying)
drop cascades to table addresses
drop cascades to table car_settings
drop cascades to table cars
drop cascades to table charges
drop cascades to table charging_processes
drop cascades to table drives
drop cascades to table geofences
drop cascades to table positions
drop cascades to table schema_migrations
drop cascades to table settings
drop cascades to table states
drop cascades to table tokens
drop cascades to table updates
DROP SCHEMA
CREATE SCHEMA
CREATE EXTENSION
CREATE EXTENSION
CREATE FUNCTION

F - Imported database backup file copied to Target host (no errors) using:
docker exec -i teslamate-postgres /bin/bash -c "PGPASSWORD=<TM_DB_PASS> psql --username teslamate teslamate" < teslamate.bck
Results can be seen HERE
G - Started new teslamate container successfully
H - Accessed tm.example.com with no Tesla token prompt shown, so it means the original token imported correctly!
I - Manually changed the access paths under teslamate's Settings Page for the cars
From:

To:

J - Attempted to access the Dashboards which opened up Grafana correctly and using the new path of tm.example.com/grafana, but there is no actual data shown only errors like this:

Followed moments later by a Grafana "Failed to Fetch" Error notice:

K - Checked all containers log files and only errors were in the grafana container logs, see HERE
L - As a last resort used the pgAdmin manager tool to access the teslamate postgres:16 database, and confirmed the data imported correctly (cars were there, and so was old data), yet something is still broken!

QUESTIONS:
1 - Since the Target host had a fresh / newly deployed postgres:16 database, are the specific cleanup restore instructions outlined HERE to "Drop existing data and reinitialize" as per previous step "E2" still needed on a fresh setup before a backup file import is performed?
2 - Any other adjustments needed on the backup data file prior to importing, due to differences in the postgres:12 to postgres:16 versions?
3 - Before and After the database import, are there any other changes or database commands needed to properly display the historical data in Grafana without errors?
ANY IDEAS HOW TO FIX?
Thank you.
Expected Behavior
Grafana should render the imported data
Steps To Reproduce
See steps in description
Relevant log output
See logs links in description

Screenshots
see screenshots inline in description
Additional data
No response
Type of installation
Docker
Version
v1.29.2

1 - Since the Target host had a fresh / newly deployed postgres:16 database, are the specific cleanup restore instructions outlined HERE to "Drop existing data and reinitialize" as per previous step "E2" still needed on a fresh setup before a backup file import is performed?

Yes, thats why they are stated in the docs.

2 - Any other adjustments needed on the backup data file prior to importing, due to differences in the postgres:12 to postgres:16 versions?

As Postgres12 is outdated since years, we can not support such bug version jumps, but there had been succesful migrations in the past from other users, so it should be possible. Best ist to do the migration on old system, then import to new system.

3 - Before and After the database import, are there any other changes or database commands needed to properly display the historical data in Grafana without errors?

https://docs.teslamate.org/docs/maintenance/upgrading_postgres

Thank you for the prompt reply @JakobLichterfeld.
If it's not too much trouble, could you please be so kind as to keep this ticket Open until there is some resolution?  I spent a lot of hours troubleshooting and putting this ticket together, with all the details and logs, and it is very disconcerting to see it closed in a minute and without any resolution.
Since teslamate is about 5 years old, other users here may have run into this or similar issues with DB upgrades & migrations, therefore someone may have solutions or ideas they can share, this is why I provide so much detail. This in turn will help the teslamate community, provided the ticket is not closed before an actual resolution...
Now, just to recap:
1 - You said that the "Drop existing data and reinitialize" process is STILL needed even with importing a backup into a fresh/newly spun-up database, which is what I did, but it is good to know since that was not clear in the docs.
2 - You had a great idea of me trying to 1st do an in-place upgrade directly on the Source/old host from postgres:12 to postgres:16 which I will do, and will report back here if it worked.
3 - If the Source upgrade-in-place is successful, I'll export/backup the upgraded Source teslamate postgres:16 database, and again import it into another freshly rebuilt Target host with postgres:16, after re-doing the "Drop existing data and reinitialize" process.
Thank you.

OK, @JakobLichterfeld and @DrMichael  I have partial good news, we are closer to a solution:
1 - I managed to do an in-place postgres:12 to postgres:16 database upgrade on the Source host!
Grafana was then accessible (on it's own FQDN of grafana.example.com) and all data seems to have survived the postgres upgrade process.
All steps are detailed HERE with all outputs.
2 - I re-exported the now upgraded postgres:16 teslamate database data from the Source host and imported it into a fresh build of the Target host after it's own postgres:16 database first received the "Drop existing data and reinitialize" process.
See all steps and detailed output HERE
Sadly, while the Source upgrade was successful and that host and is now running postgres:16, the imported data on the Target host is again NOT being rendered correctly by Grafana!
This is what I get in Grafana on the Target host tm.example.com/grafana after the import of postgres:16 data:

As this is no longer a postgres old version issue, it is clearly something else, maybe related to the paths, so I am hoping there is more I can try to get this fixed given we are so close...
Maybe something/a path I can check and change in the backup file before importing it?
Your patience and any additional help would be appreciated...
Thank you.

Grafana was then accessible (on it's own FQDN of grafana.example.com) and all data seems to have survived the postgres upgrade process.

As expected it is not an issue with TeslaMate but with your new "target" system. Please provide docker-compose.yml without credentials and make sure to delete your postgres docker volume completly before the restore to get a clear starting state (as you messed around with it).

Hello @JakobLichterfeld,
Actually all this proves is that an in-place teslamate postgres:12 to postgres:16 version upgrade worked, which is great news for anyone with an older setup.
However, as described from the start, the Target system uses one FQDN instead of two for the Source, so it is different.  Adjusting teslamate to migrate from one configuration to the other should be trivial, especially since the manual describes setups for both instances.
Also, as described in detail above and per provided logs, the postgres:16 docker volume on the Target was deleted in full followed by being re-created and having the "Drop existing data and reinitialize" process applied, BEFORE the DB restore.
HERE is the Source docker-compose.yml
HERE is the Target docker-compose.yml
If ay other details are needed, please just ask.
I am hoping someone has a suggestion on how to fix this and allow the data imported into the Target from the Source to be made accessible by Grafana on the Target...
Thank you.
P.S.  If I start the Target host with a fresh postgres:16 so not to loose new tesla API events, and once this issue is resolved I attempt to import the old/original data from the Source server (with same postgres:16 version DB), can I do so without loosing the new/previous up-to-date data in the database? Basically can one add teslamate backup data into a system rather than replace everything with the older backup?

***** SOLVED, GOT IT WORKING!!!! *****
Based on the Grafana logs error that stated dial tcp: lookup **database** on 127.0.0.11:53: server misbehaving I initially thought this was a DNS resolution problem or lack of access to Internet, which I proved it was not. Then the keyword database made me think something was hardcoded in the postgres DB so I did a search but alas found no entries matching it.
Finally, I recalled that in the Source host docker-compose.yml file the references for both the teslamate: service and grafana: service called onto the postgres database using: DATABASE_HOST=database, however in my Target setup docker-config.yml I had changed the name just for the teslamate: service to teslamate-postgres while the grafana: service entry still had the name database, and therefore they did not match causing Grafana to error out.
Recommend everyone using a variable in the .env file not just for entries that are security related, but also any that are referenced by more than one service, to ensure no typos or mismatching!
P.S. - I'd still like to know if one can add teslamate backup data "on top" of existing data, rather than replace everything with the older backup?
P.P.S. - I continue to get a Grafana Failed to fetch popup error about 10-15 seconds after Garfana dashboard data is loaded, but this is also happening on the Source host (and there is nothing relevant in the Grafana logs) so maybe it is a side-effect of the postgres:12 to postgres:16 upgrade, or possibly an issue with polling the Tesla API?

P.S. - I'd still like to know if one can add teslamate backup data "on top" of existing data, rather than replace everything with the older backup?

No. Would suggest you take your source system (which is hopefully complete) and load it to your target system and then shut down source.

P.P.S. - I continue to get a Grafana Failed to fetch popup error about 10-15 seconds after Garfana dashboard data is loaded, but this is also happening on the Source host (and there is nothing relevant in the Grafana logs) so maybe it is a side-effect of the postgres:12 to postgres:16 upgrade, or possibly an issue with polling the Tesla API?

See #3982

Thanks @cwanja !
So based on the link you referenced, the last issue I am seeing, the Grafana Failed to fetch message is unrelated to the DB upgrade or the Tesla API and is internal to the current Grafana version which makes an extra call that gets blocked and devs are working on a fix for next Grafana version.

Thanks @cwanja !
So based on the link you referenced, the last issue me and others are seeing, the Grafana Failed to fetch message, is unrelated to the DB upgrade or the Tesla API and is internal to the current Grafana version which makes an extra call that gets blocked and devs are working on a fix for next Grafana version.

There is a lot of assumptions in that block of text. Anything Grafana focused is outside my purview.
Can confirm TeslaMate with non-uBlock ad blockers running is not facing Grafana pop-ups.

Can confirm TeslaMate with non-uBlock ad blockers running is not facing Grafana pop-ups.

Correct @cwanja I further tested using a Private Safari window, and no Grafana error.
Though I did notice discrepancies from the Source teslamate host to the Target host as can be seen below.
Any idea why would the plugged-in and locked as well as the Charge Limit info not be displayed for the same vehicle on the Target host?  Both hosts run the same version teslamate/postgres and the same data, since the Source was just exported and imported to Target host!

Wake the car up.

Wake the car up.

Good call @cwanja Thank you.

however in my Target setup docker-config.yml I had changed the name just for the teslamate: service to teslamate-postgres while the grafana: service entry still had the name database, and therefore they did not match causing Grafana to error out.

Glad you found the error in your docker-compose on your own. As assumed correctly, no TeslaMate issue at all

Indeed @JakobLichterfeld, thank you for pointing me in the right direction and very glad this was just an internal configuration mismatch.
Recommend having the database entry/name for the teslamarte: and postgres: services made a ${} variable in the standard configuration, in order to avoid this.  Hopefully this ticket helps others as well.
My thanks again for everyone's help.

authors: JakobLichterfeld SemoTech cwanja

repo: teslamate-org/teslamate


text:
Best Arabic Trained Data feedback

Hi everybody
Big improvement in recognizing Arabic text with heavy vowel diacritics but hope to give option to strip out the vowel diacritics in the output.
but unfortunately there is a bug in output letters map. Some non standard Arabic characters used in the output which will be okay for printing but not for storing document digitally, it will be considered as mistake because we use standard Arabic keyboard to write and search.
Some of these letters are
used         ی               ک
right one     ي            ك
I believe if we run find and replace we could bypass this issue

the same problem ?

Please check with traineddata from tessdata_fast and post whether you face same issue with both ara and Arabic traineddata.

No, the traineddata from tessdata_fast does not have this issue.

authors: Shreeshrii bmwmy smilesecond

repo: tesseract-ocr/tessdata


text:
Update Doxygen documentation

https://tesseract-ocr.github.io/index.html
was Generated on Mon Jul 20 2015 18:38:23 by  doxygen 1.8.8
Please update. Thanks!

Documentation of tesseract generated from source code by insight.io can be found at https://www.insight.io/github.com/tesseract-ocr/tesseract
Both of these are linked from
https://github.com/tesseract-ocr/tesseract/wiki/Documentation

If https://www.insight.io/github.com/tesseract-ocr/tesseract
provided the same functionality as Doxygen, then this issue can be closed.

Up to now, Insight.io only shows the files with syntax highlighting. I got an error message that code analysis failed for my projects.

Could a travis script such as this be used to automatically update doxygen documentation?
https://github.com/miloyip/rapidjson/blob/master/travis-doxygen.sh

I have sent a pull request which updates the documentation. It is possible to automatically update that documentation, but that would add a new revision for each change of the master branch. Maybe it is sufficient to update the documentation each time a release is tagged.

Thanks, @stweil . I have added to documentation page on wiki.

New documentation is also visible here.

Is that as of master branch on May 20th or relative to some tag?

It's based on the latest git master branch.

Thanks!

@stweil Hi, insight.io creator here, tesseract-ocr should work fine on insight.io with full code intelligence not just syntax highlighting. Would you mind shooting an email to chongzhe@insight.io with regarding to the errors you see, we will help you solve the issue:)
Thanks!

authors: Shreeshrii chongzhe stweil

repo: tesseract-ocr/tesseract


text:
[Enhancement]: Add method to return CosmosDb connection string for internal network

Problem
Right now, CosmosDbContainer has a string GetConnectionString() method which returns a connection string for use on the host. It would be useful if CosmosDbContainer also had a method returning a connection string usable within a network the container is attached to. Ideally, this method would work before the container is started so that it can be used to build other containers which rely on this connection string.
Solution
I propose adding a method to class CosmosDbContainer which is functionally equivalent to:
private string GetInternalConnectionString(string networkAlias)
{
    var properties = new Dictionary<string, string>
    {
        ["AccountEndpoint"] = new UriBuilder(Uri.UriSchemeHttps, networkAlias, CosmosDbBuilder.CosmosDbPort).ToString(),
        ["AccountKey"] = CosmosDbBuilder.DefaultAccountKey
    };
    return string.Join(";", properties.Select(kvp => string.Join("=", kvp.Key, kvp.Value)));
}

Benefit
This method will make it easier to create connection strings used to build other containers which communicate with the cosmos container.
Alternatives
This connection string can be created using the code snippet proposed above, but results in duplicated code.
Would you like to help contributing this enhancement?
Yes

I do not think this refers to the Cosmos DB Emulators module only. No module shares a method or property that resolves the internal connection string, base address, endpoint, or whatever it may be called. This heavily depends on the actual test configuration and how you put various containers together. Currently, we recommend using the network alias and manually configuring the connection, as we do, for instance, with the WebDrive module or in our Flyway example. Nevertheless, I comprehend that supporting something like this might slightly reduce the amount of necessary code and configuration.

Ideally, this method would work before the container is started so that it can be used to build other containers which rely on this connection string.

I do not understand. How would you interact with the container service before it starts?

Would you have builders for each DB impl?  Something like:
var someContainer = ...;

var connectionString = new CosmosDbConnectionStringBuilder()
    .WithInternalHost(someContainer)
    .Build();

var cosmos = new CosmosDbBuilder()
    .WithConnectionString(someContainer)
    .Build();

This would be wornderful 😊

authors: HofmeisterAn goldsam

repo: testcontainers/testcontainers-dotnet


text:
Add some DEBUG loggers for connection strategies

I spent some time figuring out why I got an SSLException when trying to make my test container to work.
It appears to be a simple library version conflict on bouncycaste, but the exception cause is not logger at all.
It should be possible to log the exception causes when log level is debug.

Good idea - this would be useful. As we're suggesting in the docs that people set root log level to INFO, this shouldn't have an effect on log noise for normal usage.
Thanks for this!

authors: mbaechler rnorth

repo: testcontainers/testcontainers-java

license: mit


text:
WIP: Update to CircleCI 2.0

To respond to this warning:

It seems that we've been using almost-compatible 2.0 CircleCI config files for a while, but until the config is relocated to .circle/config.yml the change is not complete. Until we do this, our CI builds are running with reduced parallelism of 1.
Docs here

Is there any WIP left?

I just need to see the build run and pass!
Sent with GitHawk

authors: kiview rnorth


text:
Removing link to roadmap

The docs/ROADMAP.md link doesn't work and I don't see a replacement, so either remove the link or update it.

Thanks for the contribution @Spedge 😄

authors: Spedge rnorth


text:
Postgresql-backed store

See this set of .sql for inspiration on how to use fulltext:
https://github.com/minorg/thingsworth/tree/develop/java/lib/src/main/resources/db/migration

Also see the manual page:
https://www.postgresql.org/docs/9.5/textsearch.html

As an aside, we could probably get away with NOT having full-text on a first pass, per #257.

The migrations in the directory linked in the description are for Flyway. There's a Play plugin for Flyway.

On the Scala/Java side I've used jOOQ, which is a Java library but has a good Scala interface.
Slick is the more typical Scala library.

Am going through the Slick documentation, looks very neat

repo: tetherless-world/mcs-apps


text:
Cannot load .elf onto STM32F0-Discovery

I just got the new discovery board (STM32F0-Discovery) which has an STM32F051 chip on it. It still uses the ST-LINK version 2, and I can get st-util to connect. From the arm gdb I can continue and halt the firmware on the board, but I cannot load the blink example. It gives me "Load Failed".
I did alter the Makefile and main.c of the blink example to include the compiler flags and register mapping for this chip. I think the issue is with st-util talking to the chip. Below you'll find outputs from it, as well as the GDB session. It looks like the chip id is reading 00000000 which is a problem and I'm wondering if there is memory mapping issue? I don't mind lending a hand to bring support for this chip but I'm at a loss as to what to do. Any advice? Thanks!
st-util output:
mike@krusty:~/compile/stlink.git$ ./st-util
2012-05-30T21:50:25 INFO src/stlink-common.c: Loading device parameters....
2012-05-30T21:50:25 WARN src/stlink-common.c: unknown chip id! 0
Chip ID is 00000000, Core ID is  0bb11477.
KARL - should read back as 0x03, not 60 02 00 00
init watchpoints
Listening at *:4242...
GDB connected.
recv: qSupported:multiprocess+;qRelocInsn+
query: Supported;multiprocess+;qRelocInsn+
send: PacketSize=3fff;qXfer:memory-map:read+
recv: !
send: OK
recv: Hg0
send:
recv: ?
send: S05
recv: Hc-1
send:
recv: qC
send:
recv: qAttached
query: Attached;
send:
recv: g
send: 060000003817000018000000000000c00a000000010000000100000038170000ffffffffffffffffffffffffffffffffffffffffe80300206f010008f6010008
recv: qXfer:memory-map:read::0,fff
query: Xfer;memory-map:read::0,fff
Xfer: type:memory-map;op:read;annex:;addr:0;length:4095
send: m<?xml version="1.0"?><!DOCTYPE memory-map PUBLIC "+//IDN gnu.org//DTD GDB Memory Map V1.0//EN"     "http://sourceware.org/gdb/gdb-memory-map.dtd"><memory-map>  <memory type="rom" start="0x00000000" length="0x0"/>  <memory type="ram" start="0x20000000" length="0x0"/>  <memory type="flash" start="0x08000000" length="0x0">    <property name="blocksize">0x0</property>  </memory>  <memory type="ram" start="0x40000000" length="0x1fffffff"/>  <memory type="ram" start="0xe0000000" length="0x1fffffff"/>  <memory type="rom" start="0x00000000" length="0x0"/>  <memory type="rom" start="0x1ffff800" length="0x10"/></memory-map>
recv: qXfer:memory-map:read::26d,d92
query: Xfer;memory-map:read::26d,d92
Xfer: type:memory-map;op:read;annex:;addr:621;length:3474
send: l
recv: m80001f6,4
send: a1483817
recv: qTStatus
query: TStatus;
send:
recv: vKill;a410
send: OK
recv: ?
send: OK
cannot recv: -2
mike@krusty:~/compile/stlink.git$

arm-none-eabi-gdb output:
mike@krusty:~/compile/stlink.git/example/blink$ arm-none-eabi-gdb
GNU gdb (Sourcery G++ Lite 2011.03-42) 7.2.50.20100908-cvs
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=i686-pc-linux-gnu --target=arm-none-eabi".
For bug reporting instructions, please see:
<https://support.codesourcery.com/GNUToolchain/>.
(gdb) tar ext :4242
Remote debugging using :4242
0x080001f6 in ?? ()
(gdb) kill
Kill the program being debugged? (y or n) y
(gdb) load blink_F0.elf
Loading section .text, size 0x48 lma 0x20000000
Load failed
(gdb) disconnect
Ending remote debugging.
(gdb)

diff of the blink example files:
diff --git a/example/blink/Makefile b/example/blink/Makefile
index cf7221d..60fab84 100644
--- a/example/blink/Makefile
+++ b/example/blink/Makefile
@@ -12,8 +12,9 @@ DEF_CFLAGS+=-Wl,-Ttext,0x20000000 -Wl,-e,0x20000000
 CFLAGS_VL=$(DEF_CFLAGS) -mcpu=cortex-m3 -DCONFIG_STM32VL_DISCOVERY=1
 CFLAGS_L=$(DEF_CFLAGS) -mcpu=cortex-m3 -DCONFIG_STM32L_DISCOVERY
 CFLAGS_F4=$(DEF_CFLAGS) -mcpu=cortex-m4 -DCONFIG_STM32F4_DISCOVERY=1
+CFLAGS_F0=$(DEF_CFLAGS) -mcpu=cortex-m0 -DCONFIG_STM32F0_DISCOVERY=1

-all: blink_32VL.elf blink_32L.elf blink_F4.elf
+all: blink_32VL.elf blink_32L.elf blink_F4.elf blink_F0.elf

 %.bin: %.elf
    $(OBJCOPY) -O binary $^ $@
@@ -24,6 +25,8 @@ blink_32L.elf: main.c
    $(CC) $(CFLAGS_L) $^ -o $@
 blink_F4.elf: main.c
    $(CC) $(CFLAGS_F4) $^ -o $@
+blink_F0.elf: main.c
+   $(CC) $(CFLAGS_F0) $^ -o $@

 clean:
    rm -rf *.elf
diff --git a/example/blink/main.c b/example/blink/main.c
index b70a136..26fcca0 100644
--- a/example/blink/main.c
+++ b/example/blink/main.c
@@ -58,6 +58,26 @@ static inline void setup_leds(void)
    (1 << (13 * 2)) | (1 << (14 * 2)) | (1 << (15 * 2));
 }

+#elif CONFIG_STM32F0_DISCOVERY
+
+#define GPIOC 0x48000800 /* port C */
+#define GPIOC_MODER (GPIOC + 0x00) /* port mode register */
+#define LED_PORT_ODR (GPIOC + 0x14) /* port output data register */
+
+#define LED_BLUE (1 << 8) /* port C, pin 8 */
+#define LED_GREEN (1 << 9) /* port C, pin 9 */
+#define LED_ORANGE 0
+#define LED_RED 0
+
+void _tmain(void) {
+   main();
+}
+static inline void setup_leds(void)
+{
+  /* configure port 8 and 9 as output */
+  *(volatile uint32_t*)GPIOC_MODER |= (1 << (9* 2)) | (1 << (8 * 2));
+}
+
 #else
 #error "Architecture must be defined!"
 #endif /* otherwise, error */

Fixed in #86.

repo: texane/stlink


text:
Angular 2 NgModule Support

Trying to import MaskedInput and declare it in the NgModule decorator, like in the readme, gives me a TypeError.

@Ckearns1210 Could you please confirm what version of angular2-text-mask you're using?

repo: text-mask/text-mask

license: Unlicense