text:
denseflow compile problem

it seeems like denseflow needs to be compiled with OpenCV 2.4.13 and CUDA. However this version of OpenCV does not match with CUDA9.0+... And I can't compile OpenCV successfully. Is there any solution?  This is a CentOS 7.6, cmake 3.15.0, gcc 6.5.0, Titan X.

Hi @aaab8b
After several workaround, dense_flow is now supporting the latest OpenCV 4.1.0 (https://github.com/yjxiong/dense_flow/pull/4)! Detailed steps are as follows. Note that these steps are tested on a machine with Ubuntu 18.04 + CUDA 9.2/10.0. Please report if you encounter any further problem.

(For CUDA 10.0 only) CUDA 9.x should have no problem.
Video decoder is deprecated in CUDA 10.0. To handle this, download NVIDIA VIDEO CODEC SDK and copy the header files to your cuda path (/usr/local/cuda-10.0/include/ for example). Note that you may have to do as root.

unzip Video_Codec_SDK_9.0.20.zip
cp Video_Codec_SDK_9.0.20/include/nvcuvid.h /usr/local/cuda-10.0/include/
cp Video_Codec_SDK_9.0.20/include/cuviddec.h /usr/local/cuda-10.0/include/

Install OpenCV 4.1.0
Note that I didn't come across any other prerequisites for installation. If any, please kindly remind me to add in the comment.

(a) Obtain the OpenCV 4.1.0 and its extra modules (optflow. etc.) by
# assume you are now in the directory of /PATH/TO/MMACTION/third_party/
wget -O OpenCV-4.1.0.zip wget https://github.com/opencv/opencv/archive/4.1.0.zip
unzip OpenCV-4.1.0.zip
wget -O OpenCV_contrib-4.1.0.zip https://github.com/opencv/opencv_contrib/archive/4.1.0.zip
unzip OpenCV_contrib-4.1.0.zip

(b) Install OpenCV 4.1.0 by
cd opencv-4.1.0
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -DWITH_CUDA=ON -DOPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-4.1.0/modules/ -DWITH_TBB=ON -DBUILD_opencv_cnn_3dobj=OFF -DBUILD_opencv_dnn=OFF -DBUILD_opencv_dnn_modern=OFF -DBUILD_opencv_dnns_easily_fooled=OFF ..
make -j

Note that remember to explicitly setting -DOPENCV_ENABLE_NONFREE=ON if you want to enable warped_flow (warped flow is proposed in TSN).

Install dense_flow

cd ../../dense_flow
git checkout opencv-4.1.0 # switch to the branch for opencv-4.1.0
mkdir build && cd build
make -j

@zhaoyue-zephyrus
git checkout opencv-4.1.0 #the branch is invalid now.
use the master branch,get the error:
fatal error: opencv2/xfeatures2d.hpp: No such file or directory
#include "opencv2/xfeatures2d.hpp"
^~~~~~~~~~~~~~~~~~~~~~~~~
But my opencv is installed.
.
├── decord
│   ├── 3rdparty
│   ├── cmake
│   ├── CMakeLists.txt
│   ├── docs
│   ├── examples
│   ├── include
│   ├── LICENSE
│   ├── python
│   ├── README.md
│   ├── src
│   ├── tests
│   └── tools
├── dense_flow
│   ├── build
│   ├── build_of.py
│   ├── cmake
│   ├── CMakeCache.txt
│   ├── CMakeFiles
│   ├── cmake_install.cmake
│   ├── CMakeLists.txt
│   ├── include
│   ├── LICENSE
│   ├── Makefile
│   ├── matlab
│   ├── README.md
│   ├── src
│   └── tools
├── libzip-1.5.2
│   ├── API-CHANGES.md
│   ├── appveyor.yml
│   ├── AUTHORS
│   ├── build
│   ├── cmake-config.h.in
│   ├── CMakeLists.txt
│   ├── cmake-zipconf.h.in
│   ├── examples
│   ├── FindNettle.cmake
│   ├── INSTALL.md
│   ├── lib
│   ├── libzip.pc.in
│   ├── LICENSE
│   ├── man
│   ├── NEWS.md
│   ├── README.md
│   ├── regress
│   ├── src
│   ├── THANKS
│   └── TODO.md
├── opencv-4.1.0
│   ├── 3rdparty
│   ├── apps
│   ├── build
│   ├── cmake
│   ├── CMakeLists.txt
│   ├── CONTRIBUTING.md
│   ├── data
│   ├── doc
│   ├── include
│   ├── LICENSE
│   ├── modules
│   ├── platforms
│   ├── README.md
│   └── samples
└── opencv_contrib-4.1.0
├── CONTRIBUTING.md
├── doc
├── LICENSE
├── modules
├── README.md
└── samples

Hi @zhangluustb
May I know your command for installing opencv? Seems that xfeatures2d is not installed properly. Make sure that your are installing with open_contrib by specifying -DOPENCV_EXTRA_MODULES_PATH=....

@zhaoyue-zephyrus thank you for your reply,this error caused by that i have two versions of opencv,I fixed this error by modify the findPackge of opencv.
But anothor error occur:
./extract_gpu -f=/home/zhanglu/Desktop/mmaction/data/ucf101/videos/Biking/v_Biking_g04_c03.avi -x=tmp/flow_x -y=tmp/flow_y -i=tmp/image -b=20 -t=1 -d=0 -s=1 -o=dir
2019-06-27 18:30:41,145 FATAL [default] CRASH HANDLED; Application has crashed due to [SIGSEGV] signal
2019-06-27 18:30:41,145 WARN  [default] Aborting application. Reason: Fatal log at [/home/zhanglu/Desktop/mmaction/third_party/dense_flow/include/easylogging++/src/easylogging++.h:5583]
Aborted (core dumped)

@zhangluustb  Good to hear that :)
As of the new error, it arises probably because you don't have sub-directory named tmp/ while trying to save images and optical flows to tmp/. The c++ interface of dense_flow cannot create new directory automatically. I will add this in the new README.

@zhaoyue-zephyrus
./extract_warp_gpu -f test.avi -x tmp/flow_x -y tmp/flow_y -i tmp/image -b 20 -t 1 -d 0 -s 1 -o dir
2019-06-27 19:19:43,986 FATAL [default] Check failed: [video_stream.isOpened()] Cannot open video stream "true" for optical flow extraction.
2019-06-27 19:19:43,986 WARN  [default] Aborting application. Reason: Fatal log at [/home/zhanglu/Desktop/mmaction/third_party/dense_flow/src/dense_warp_flow_gpu.cpp:30]
Aborted (core dumped)
It's so wired.help me

@zhangluustb
Don't panic. Could you try the following script?
mkdir tmp/
./extract_warp_gpu -f=test.avi -x=tmp/flow_x -y=tmp/flow_y -i=tmp/image -b=20 -t=1 -d=0 -s=1 -o dir

Also, note that you have to build opencv-4.1.0 with -DOPENCV_ENABLE_NONFREE=ON.

i still can't make dense_flow correctly... I successfully installed opencv-4.1.0 with your guide, However, when i make dense_flow, it comes to some errors:
/data/bingzd/mmaction/third_party/dense_flow/include/common.h:23:20: 错误：‘vector’未声明
vector& encoded_x, vector& encoded_y,
^
/data/bingzd/mmaction/third_party/dense_flow/include/common.h:23:26: 错误：expected ‘,’ or ‘...’ before ‘<’ token
vector& encoded_x, vector& encoded_y,
^
/data/bingzd/mmaction/third_party/dense_flow/include/common.h:36:18: 错误：变量或字段‘writeImages’声明为 void
void writeImages(vector<vector> images, string name_temp);
^
/data/bingzd/mmaction/third_party/dense_flow/include/common.h:36:18: 错误：‘vector’在此作用域中尚未声明
/data/bingzd/mmaction/third_party/dense_flow/include/common.h:36:18: 附注：建议的替代：
In file included from /usr/include/c++/4.8.2/vector:64:0,
from /usr/include/c++/4.8.2/bits/random.h:34,
from /usr/include/c++/4.8.2/random:50,
from /usr/include/c++/4.8.2/bits/stl_algo.h:65,
from /usr/include/c++/4.8.2/algorithm:62,
from /home/bingzd/opencv-4.1/include/opencv4/opencv2/core/base.hpp:55,
from /home/bingzd/opencv-4.1/include/opencv4/opencv2/core.hpp:54,
from /home/bingzd/opencv-4.1/include/opencv4/opencv2/video/tracking.hpp:47,
from /data/bingzd/mmaction/third_party/dense_flow/include/common.h:10,
from /data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:5:
/usr/include/c++/4.8.2/bits/stl_vector.h:210:11: 附注：  ‘std::vector’
class vector : protected _Vector_base<_Tp, _Alloc>
^
In file included from /data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:5:0:
/data/bingzd/mmaction/third_party/dense_flow/include/common.h:36:25: 错误：‘vector’在此作用域中尚未声明
void writeImages(vector<vector> images, string name_temp);
^
/data/bingzd/mmaction/third_party/dense_flow/include/common.h:36:25: 附注：建议的替代：
In file included from /usr/include/c++/4.8.2/vector:64:0,
from /usr/include/c++/4.8.2/bits/random.h:34,
from /usr/include/c++/4.8.2/random:50,
from /usr/include/c++/4.8.2/bits/stl_algo.h:65,
from /usr/include/c++/4.8.2/algorithm:62,
from /home/bingzd/opencv-4.1/include/opencv4/opencv2/core/base.hpp:55,
from /home/bingzd/opencv-4.1/include/opencv4/opencv2/core.hpp:54,
from /home/bingzd/opencv-4.1/include/opencv4/opencv2/video/tracking.hpp:47,
from /data/bingzd/mmaction/third_party/dense_flow/include/common.h:10,
from /data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:5:
/usr/include/c++/4.8.2/bits/stl_vector.h:210:11: 附注：  ‘std::vector’
class vector : protected _Vector_base<_Tp, _Alloc>
^
In file included from /data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:5:0:
/data/bingzd/mmaction/third_party/dense_flow/include/common.h:36:37: 错误：expected primary-expression before ‘>>’ token
void writeImages(vector<vector> images, string name_temp);
^
/data/bingzd/mmaction/third_party/dense_flow/include/common.h:36:40: 错误：‘images’在此作用域中尚未声明
void writeImages(vector<vector> images, string name_temp);
^
/data/bingzd/mmaction/third_party/dense_flow/include/common.h:36:48: 错误：‘string’在此作用域中尚未声明
void writeImages(vector<vector> images, string name_temp);
^
/data/bingzd/mmaction/third_party/dense_flow/include/common.h:36:48: 附注：建议的替代：
In file included from /usr/include/c++/4.8.2/string:39:0,
from /usr/include/c++/4.8.2/stdexcept:39,
from /usr/include/c++/4.8.2/array:38,
from /home/bingzd/opencv-4.1/include/opencv4/opencv2/core/cvdef.h:671,
from /home/bingzd/opencv-4.1/include/opencv4/opencv2/core.hpp:52,
from /home/bingzd/opencv-4.1/include/opencv4/opencv2/video/tracking.hpp:47,
from /data/bingzd/mmaction/third_party/dense_flow/include/common.h:10,
from /data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:5:
/usr/include/c++/4.8.2/bits/stringfwd.h:62:33: 附注：  ‘std::string’
typedef basic_string    string;
^
/data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:33:20: 错误：‘vector’未声明
vector& encoded_x, vector& encoded_y,
^
/data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:33:26: 错误：expected ‘,’ or ‘...’ before ‘<’ token
vector& encoded_x, vector& encoded_y,
^
/data/bingzd/mmaction/third_party/dense_flow/src/common.cpp: 在函数‘void encodeFlowMap(const cv::Mat&, const cv::Mat&, int)’中:
/data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:39:25: 错误：‘bound’在此作用域中尚未声明
-bound, bound);
^
/data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:41:9: 错误：‘to_jpg’在此作用域中尚未声明
if (to_jpg) {
^
/data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:42:38: 错误：‘encoded_x’在此作用域中尚未声明
imencode(".jpg", flow_img_x, encoded_x);
^
/data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:43:38: 错误：‘encoded_y’在此作用域中尚未声明
imencode(".jpg", flow_img_y, encoded_y);
^
/data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:45:9: 错误：‘encoded_x’在此作用域中尚未声明
encoded_x.resize(flow_img_x.total());
^
/data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:46:9: 错误：‘encoded_y’在此作用域中尚未声明
encoded_y.resize(flow_img_y.total());
^
/data/bingzd/mmaction/third_party/dense_flow/src/common.cpp: 在全局域：
/data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:52:18: 错误：变量或字段‘writeImages’声明为 void
void writeImages(vector<vector> images, string name_temp){
^
/data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:52:18: 错误：‘vector’在此作用域中尚未声明
/data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:52:18: 附注：建议的替代：
In file included from /usr/include/c++/4.8.2/vector:64:0,
from /usr/include/c++/4.8.2/bits/random.h:34,
from /usr/include/c++/4.8.2/random:50,
from /usr/include/c++/4.8.2/bits/stl_algo.h:65,
from /usr/include/c++/4.8.2/algorithm:62,
from /home/bingzd/opencv-4.1/include/opencv4/opencv2/core/base.hpp:55,
from /home/bingzd/opencv-4.1/include/opencv4/opencv2/core.hpp:54,
from /home/bingzd/opencv-4.1/include/opencv4/opencv2/video/tracking.hpp:47,
from /data/bingzd/mmaction/third_party/dense_flow/include/common.h:10,
from /data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:5:
/usr/include/c++/4.8.2/bits/stl_vector.h:210:11: 附注：  ‘std::vector’
class vector : protected _Vector_base<_Tp, _Alloc>
^
/data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:52:37: 错误：expected primary-expression before ‘>>’ token
void writeImages(vector<vector> images, string name_temp){
^
/data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:52:40: 错误：‘images’在此作用域中尚未声明
void writeImages(vector<vector> images, string name_temp){
^
/data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:52:48: 错误：‘string’在此作用域中尚未声明
void writeImages(vector<vector> images, string name_temp){
^
/data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:52:48: 附注：建议的替代：
In file included from /usr/include/c++/4.8.2/string:39:0,
from /usr/include/c++/4.8.2/stdexcept:39,
from /usr/include/c++/4.8.2/array:38,
from /home/bingzd/opencv-4.1/include/opencv4/opencv2/core/cvdef.h:671,
from /home/bingzd/opencv-4.1/include/opencv4/opencv2/core.hpp:52,
from /home/bingzd/opencv-4.1/include/opencv4/opencv2/video/tracking.hpp:47,
from /data/bingzd/mmaction/third_party/dense_flow/include/common.h:10,
from /data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:5:
/usr/include/c++/4.8.2/bits/stringfwd.h:62:33: 附注：  ‘std::string’
typedef basic_string    string;
^
make[2]: *** [CMakeFiles/denseflow.dir/src/common.cpp.o] 错误 1
make[1]: *** [CMakeFiles/denseflow.dir/all] 错误 2
make: *** [all] 错误 2
my environment is CentOS 7.6, CUDA 9.0 gcc-5.4.0,Titan X.

@zhaoyue-zephyrus
is my opencv environment variables setting not correct?  Since I don't have the root permission so i install opencv with -DCMAKE_INSTALL_PREFIX to another location instead of the default location. Then how should i set the enviroment variables...
And thank you so much for your help...

@zhangluustb
Don't panic. Could you try the following script?
mkdir tmp/
./extract_warp_gpu -f=test.avi -x=tmp/flow_x -y=tmp/flow_y -i=tmp/image -b=20 -t=1 -d=0 -s=1 -o dir

Also, note that you have to build opencv-4.1.0 with -DOPENCV_ENABLE_NONFREE=ON.

It works,Thks for u help.

-DOPENCV_ENABLE_NONFREE=ON is essential.
in the command line "=" is essential.

@aaab8b
Seems that you are actually using gcc-4.8.2 instead of gcc-5.4.0 as you claimed? Please check.
To specify the opencv path, do OpenCV_DIR=/PATH/TO/YOUR/OPENCV4/build cmake ... Remember to remove the cmake cache or more simply rm build/ before re-installing.

@zhaoyue-zephyrus
thank you for your help, and i rechecked my gcc, re-make the opencv and do the dense flow cmake with your command.however it still get some errors like this...
In file included from /data/bingzd/mmaction/third_party/opencv-4.1.0/modules/core/include/opencv2/core.hpp:3288:0,                                                        [1879/1879]
from /data/bingzd/mmaction/third_party/opencv-4.1.0/modules/video/include/opencv2/video/tracking.hpp:47,
from /data/bingzd/mmaction/third_party/dense_flow/include/common.h:10,
from /data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:5:
/data/bingzd/mmaction/third_party/opencv-4.1.0/modules/core/include/opencv2/core/operations.hpp:426:12: 错误：‘String’不是一个类型名
CV_EXPORTS String format( const char* fmt, ... ) CV_FORMAT_PRINTF(1, 2);
^
In file included from /home/bingzd/GCC-5.4.0/include/c++/5.4.0/bits/ios_base.h:41:0,
from /home/bingzd/GCC-5.4.0/include/c++/5.4.0/ios:42,
from /home/bingzd/GCC-5.4.0/include/c++/5.4.0/istream:38,
from /home/bingzd/GCC-5.4.0/include/c++/5.4.0/sstream:38,
from /home/bingzd/GCC-5.4.0/include/c++/5.4.0/complex:45,
from /data/bingzd/mmaction/third_party/opencv-4.1.0/modules/core/include/opencv2/core/cvstd.inl.hpp:47,
from /data/bingzd/mmaction/third_party/opencv-4.1.0/modules/core/include/opencv2/core.hpp:3289,
from /data/bingzd/mmaction/third_party/opencv-4.1.0/modules/video/include/opencv2/video/tracking.hpp:47,
from /data/bingzd/mmaction/third_party/dense_flow/include/common.h:10,
from /data/bingzd/mmaction/third_party/dense_flow/src/common.cpp:5:
/home/bingzd/GCC-5.4.0/include/c++/5.4.0/bits/locale_classes.h:163:18: 错误：‘string’不是命名空间‘std’中的一个类型名
locale(const std::string& __s) : locale(__s.c_str()) { }
^
/home/bingzd/GCC-5.4.0/include/c++/5.4.0/bits/locale_classes.h:177:40: 错误：‘string’不是命名空间‘std’中的一个类型名
locale(const locale& __base, const std::string& __s, category __cat)
^
/home/bingzd/GCC-5.4.0/include/c++/5.4.0/bits/locale_classes.h:243:5: 错误：‘_GLIBCXX_DEFAULT_ABI_TAG’不是一个类型名
_GLIBCXX_DEFAULT_ABI_TAG
^
/home/bingzd/GCC-5.4.0/include/c++/5.4.0/bits/locale_classes.h: 在构造函数‘std::locale::locale(const int&)’中:
/home/bingzd/GCC-5.4.0/include/c++/5.4.0/bits/locale_classes.h:163:49: 错误：对成员‘c_str’的请求出现在‘__s’中，而后者具有非类类型‘const int’
locale(const std::string& __s) : locale(__s.c_str()) { }
^
/home/bingzd/GCC-5.4.0/include/c++/5.4.0/bits/locale_classes.h: 在构造函数‘std::locale::locale(const std::locale&, const int&, std::locale::category)’中:
/home/bingzd/GCC-5.4.0/include/c++/5.4.0/bits/locale_classes.h:178:26: 错误：对成员‘c_str’的请求出现在‘__s’中，而后者具有非类类型‘const int’
: locale(__base, __s.c_str(), __cat) { }
^
/home/bingzd/GCC-5.4.0/include/c++/5.4.0/bits/locale_classes.h: 在全局域：
/home/bingzd/GCC-5.4.0/include/c++/5.4.0/bits/locale_classes.h:633:44: 错误：expected initializer before ‘:’ token
class _GLIBCXX_NAMESPACE_CXX11 collate : public locale::facet
^
/home/bingzd/GCC-5.4.0/include/c++/5.4.0/bits/locale_classes.h:784:23: 错误：expected initializer before ‘<’ token
locale::id collate<_CharT>::id;
^
/home/bingzd/GCC-5.4.0/include/c++/5.4.0/bits/locale_classes.h:789:12: 错误：expected initializer before ‘<’ token
collate::_M_compare(const char*, const char*) const throw();
^
/home/bingzd/GCC-5.4.0/include/c++/5.4.0/bits/locale_classes.h:793:12: 错误：expected initializer before ‘<’ token
collate::_M_transform(char*, const char*, size_t) const throw();
^
/home/bingzd/GCC-5.4.0/include/c++/5.4.0/bits/locale_classes.h:798:12: 错误：expected initializer before ‘<’ token
collate<wchar_t>::_M_compare(const wchar_t*, const wchar_t*) const throw();
^
/home/bingzd/GCC-5.4.0/include/c++/5.4.0/bits/locale_classes.h:802:12: 错误：expected initializer before ‘<’ token
collate<wchar_t>::_M_transform(wchar_t*, const wchar_t*, size_t) const throw();
^
/home/bingzd/GCC-5.4.0/include/c++/5.4.0/bits/locale_classes.h:807:51: 错误：expected initializer before ‘:’ token

dense_flow downloaded with mmaction by the following command
git clone --recursive https://github.com/open-mmlab/mmaction.git
seems not to be the correct branch for OpenCV 4.1.0 (maybe for OpenCV 2.4.13?) and result in compile errors with no opencv2/gpu/gpu.hpp.
When I clone directly the dense_flow repository by the following command, compilation succeeded.
git clone --recursive http://github.com/yjxiong/dense_flow
I would appreciate if the issue would be fixed.
Thank you for publishing the fascinating framework.

dense_flow downloaded with mmaction by the following command
git clone --recursive https://github.com/open-mmlab/mmaction.git
seems not to be the correct branch for OpenCV 4.1.0 (maybe for OpenCV 2.4.13?) and result in compile errors with no opencv2/gpu/gpu.hpp.
When I clone directly the dense_flow repository by the following command, compilation succeeded.
git clone --recursive http://github.com/yjxiong/dense_flow
I would appreciate if the issue would be fixed.
Thank you for publishing the fascinating framework.

@acrophile  Thanks for pointing out this issue. This is because I was then waiting for Yuanjun to merge my PRs. Now the submodule has been updated with the master branch. You might check as well.

Hi @aaab8b
Sorry I don't have any CentOS system to exactly reproduce this issue. Are you having the latest denseflow? Several changes have been made to support opencv-4.1 and please check the commits. It could be that you are still having the master branch originally for opencv-2.4

@zhaoyue-zephyrus Thanks a lot... I changed it to the latest denseflow and it worked just fine.

Hi @zhaoyue-zephyrus  I am trying to use denseflow for extracting optical flow for my own data. I have successfully build opencv  4.1.0 with cuda 9.2 and have build the denseflow repo also. However when I run the command mkdir tmp/ ./extract_warp_gpu -f=test.avi -x=tmp/flow_x -y=tmp/flow_y -i=tmp/image -b=20 -t=1 -d=0 -s=1 -o dir from the build directory of denseflow it is just processing for a long time (without any output) and on nvidia-smi the process is also utilizing the GPU however there is nothing getting downloaded in the tmp folder nor any logs in the myeasylog.log. I am also not sure if in -f parser command I can pass the path to the video or the video needs to be present in the build directory of denseflow itself, also my videos are in .mp4 format is it necessary for them to be in .avi?
This is the output after building denseflow.

Can you please help me with this, I have been following this issue since a long while. Thank you very much.

Hi @zhaoyue-zephyrus  I got the output downloaded to the tmp folder. But is the code very slow with the GPU as well? I was previously trying to extract with CPU and since the tvl1 algorithm was taking lot of time,  I decided to use denseflow repo for the GPU support.

Hello @rishabh2301 ,
I also noticed the same. I had a clip having 1482 frames, and it took really long time to extract the optical flow (type=tvl1). I used the python script build_rawframes.py in data_tools directory. Though it was using gpu, but it seemed that only a small amount of memory was being used by the process even though I had two gpus each with lot of memory left.

Hi @atique81 , yes it is taking time with GPU as well. The only thing that can be done is increasing the number of workers in the script build_rawframes.py for multiprocessing according to the architecture of your system. I am also facing the same issue as you.

@zhaoyue-zephyrus  Hello, I do as described, but I get an error as follows:
VIDIOC_REQBUFS: Inappropriate ioctl for device
2019-12-13 21:45:16,650 FATAL [default] Check failed: [video_stream.isOpened()] Cannot open video stream "test.avi" for optical flow extraction.
2019-12-13 21:45:16,650 WARN  [default] Aborting application. Reason: Fatal log at [/home/l/dense_flow/src/dense_flow_gpu.cpp:19]
Can you give me some suggestions to solve it? Thanks.

OpenCV_DIR=../../opencv-4.1.0/build/  cmake ..

Hi @zhaoyue-zephyrus , I followed your instruction to set up the dense_flow. But when I make -j the dense_flow, I got the following error.
/mmaction/third_party/dense_flow/src/dense_flow.cpp: In function ‘void calcDenseFlow(std::__cxx11::string, int, int, int, std::vector<std::vector<unsigned char> >&, std::vector<std::vector<unsigned char> >&, std::vector<std::vector<unsigned char> >&)’: /mmaction/third_party/dense_flow/src/dense_flow.cpp:20:17: error: ‘DualTVL1OpticalFlow’ is not a member of ‘cv’      cv::Ptr<cv::DualTVL1OpticalFlow> alg_tvl1 = cv::createOptFlow_DualTVL1();                  ^~~~~~~~~~~~~~~~~~~ /mmaction/third_party/dense_flow/src/dense_flow.cpp:20:17: note: suggested alternative: ‘DISOpticalFlow’      cv::Ptr<cv::DualTVL1OpticalFlow> alg_tvl1 = cv::createOptFlow_DualTVL1();                  ^~~~~~~~~~~~~~~~~~~                  DISOpticalFlow /mmaction/third_party/dense_flow/src/dense_flow.cpp:20:17: error: ‘DualTVL1OpticalFlow’ is not a member of ‘cv’ /mmaction/third_party/dense_flow/src/dense_flow.cpp:20:17: note: suggested alternative: ‘DISOpticalFlow’      cv::Ptr<cv::DualTVL1OpticalFlow> alg_tvl1 = cv::createOptFlow_DualTVL1();                  ^~~~~~~~~~~~~~~~~~~                  DISOpticalFlow /mmaction/third_party/dense_flow/src/dense_flow.cpp:20:36: error: template argument 1 is invalid      cv::Ptr<cv::DualTVL1OpticalFlow> alg_tvl1 = cv::createOptFlow_DualTVL1();                                     ^ /mmaction/third_party/dense_flow/src/dense_flow.cpp:20:53: error: ‘createOptFlow_DualTVL1’ is not a member of ‘cv’      cv::Ptr<cv::DualTVL1OpticalFlow> alg_tvl1 = cv::createOptFlow_DualTVL1();                                                      ^~~~~~~~~~~~~~~~~~~~~~ /mmaction/third_party/dense_flow/src/dense_flow.cpp:32:45: error: ‘CV_BGR2GRAY’ was not declared in this scope              cvtColor(prev_image, prev_gray, CV_BGR2GRAY);                                              ^~~~~~~~~~~ /mmaction/third_party/dense_flow/src/dense_flow.cpp:37:51: error: ‘CV_BGR2GRAY’ was not declared in this scope              cvtColor(capture_image, capture_gray, CV_BGR2GRAY);                                                    ^~~~~~~~~~~ /mmaction/third_party/dense_flow/src/dense_flow.cpp:47:29: error: base operand of ‘->’ is not a pointer                      alg_tvl1->calc(prev_gray, capture_gray, flow);                              ^~ /mmaction/third_party/dense_flow/src/dense_flow_gpu.cpp: In function ‘void calcDenseFlowGPU(std::__cxx11::string, int, int, int, int, std::vector<std::vector<unsigned char> >&, std::vector<std::vector<unsigned char> >&, std::vector<std::vector<unsigned char> >&, int, int)’: /mmaction/third_party/dense_flow/src/dense_flow_gpu.cpp:57:45: error: ‘CV_BGR2GRAY’ was not declared in this scope              cvtColor(prev_image, prev_gray, CV_BGR2GRAY);                                              ^~~~~~~~~~~ /mmaction/third_party/dense_flow/src/dense_flow_gpu.cpp:70:51: error: ‘CV_BGR2GRAY’ was not declared in this scope              cvtColor(capture_image, capture_gray, CV_BGR2GRAY);                                                    ^~~~~~~~~~~ /mmaction/third_party/dense_flow/src/dense_flow_gpu.cpp: In function ‘void calcDenseFlowPureGPU(std::__cxx11::string, int, int, int, int, std::vector<std::vector<unsigned char> >&, std::vector<std::vector<unsigned char> >&, std::vector<std::vector<unsigned char> >&)’: /mmaction/third_party/dense_flow/src/dense_flow_gpu.cpp:169:45: error: ‘CV_BGR2GRAY’ was not declared in this scope              cvtColor(prev_image, prev_gray, CV_BGR2GRAY);                                              ^~~~~~~~~~~ /mmaction/third_party/dense_flow/src/dense_flow_gpu.cpp:177:51: error: ‘CV_BGR2GRAY’ was not declared in this scope              cvtColor(capture_image, capture_gray, CV_BGR2GRAY);                                                    ^~~~~~~~~~~ /mmaction/third_party/dense_flow/src/dense_warp_flow_gpu.cpp: In function ‘void calcDenseWarpFlowGPU(std::__cxx11::string, int, int, int, int, std::vector<std::vector<unsigned char> >&, std::vector<std::vector<unsigned char> >&)’: /mmaction/third_party/dense_flow/src/dense_warp_flow_gpu.cpp:68:36: error: ‘CV_BGR2GRAY’ was not declared in this scope     cvtColor(prev_image, prev_gray, CV_BGR2GRAY);                                     ^~~~~~~~~~~ /mmaction/third_party/dense_flow/src/dense_warp_flow_gpu.cpp:84:42: error: ‘CV_BGR2GRAY’ was not declared in this scope     cvtColor(capture_image, capture_gray, CV_BGR2GRAY);                                           ^~~~~~~~~~~ CMakeFiles/denseflow.dir/build.make:95: recipe for target 'CMakeFiles/denseflow.dir/src/dense_flow.cpp.o' failed make[2]: *** [CMakeFiles/denseflow.dir/src/dense_flow.cpp.o] Error 1 make[2]: *** Waiting for unfinished jobs.... CMakeFiles/denseflow.dir/build.make:108: recipe for target 'CMakeFiles/denseflow.dir/src/dense_flow_gpu.cpp.o' failed make[2]: *** [CMakeFiles/denseflow.dir/src/dense_flow_gpu.cpp.o] Error 1 CMakeFiles/denseflow.dir/build.make:121: recipe for target 'CMakeFiles/denseflow.dir/src/dense_warp_flow_gpu.cpp.o' failed make[2]: *** [CMakeFiles/denseflow.dir/src/dense_warp_flow_gpu.cpp.o] Error 1 CMakeFiles/Makefile2:211: recipe for target 'CMakeFiles/denseflow.dir/all' failed make[1]: *** [CMakeFiles/denseflow.dir/all] Error 2 Makefile:103: recipe for target 'all' failed make: *** [all] Error 2
Here is my environment.

Ubuntu 18.04
CUDA 10.2
cmake 3.17.2
opencv 4.1.0
gpu RTX 2080 Ti

I would appreciate it if you can give me some suggestions.

Hi @zhaoyue-zephyrus I have already install dense follow But when I test it，it shows that
VIDIOC_REQBUFS: Inappropriate ioctl for device
2020-10-24 05:43:11,627 FATAL [default] Check failed: [video_stream.isOpened()] Cannot open video stream "test.avi" for optical flow extraction.
2020-10-24 05:43:11,627 WARN  [default] Aborting application. Reason: Fatal log at [/data/xxy/dense_flow/src/dense_flow_gpu.cpp:19]
Environment
OpenCV 4.1.0
CUDA 10.0
Could you give me some suggestion ?Thank you.

Hi @zhaoyue-zephyrus When I compile dense_flow, it shows that, how to fix it?
Scanning dependencies of target denseflow
[  7%] Building CXX object CMakeFiles/denseflow.dir/src/zip_utils.cpp.o
[ 14%] Building CXX object CMakeFiles/denseflow.dir/src/common.cpp.o
[ 21%] Building CXX object CMakeFiles/denseflow.dir/src/dense_flow.cpp.o
[ 28%] Building CXX object CMakeFiles/denseflow.dir/src/dense_flow_gpu.cpp.o
[ 35%] Building CXX object CMakeFiles/denseflow.dir/src/dense_warp_flow_gpu.cpp.o
[ 42%] Linking CXX static library libdenseflow.a
[ 42%] Built target denseflow
Scanning dependencies of target extract_warp_gpu
Scanning dependencies of target extract_gpu
Scanning dependencies of target pydenseflow
Scanning dependencies of target extract_cpu
[ 50%] Building CXX object CMakeFiles/extract_warp_gpu.dir/tools/extract_warp_flow_gpu.cpp.o
[ 57%] Building CXX object CMakeFiles/extract_cpu.dir/tools/extract_flow.cpp.o
[ 64%] Building CXX object CMakeFiles/pydenseflow.dir/src/py_denseflow.cpp.o
[ 71%] Building CXX object CMakeFiles/extract_gpu.dir/tools/extract_flow_gpu.cpp.o
[ 78%] Linking CXX executable extract_warp_gpu
[ 85%] Linking CXX executable extract_cpu
[ 92%] Linking CXX executable extract_gpu
[ 92%] Built target extract_cpu
[ 92%] Built target extract_warp_gpu
[ 92%] Built target extract_gpu
[100%] Linking CXX shared library libpydenseflow.so
/usr/bin/ld: /usr/local/lib/libpython3.6m.a(abstract.o): relocation R_X86_64_32S against `_Py_NotImplementedStruct' can not be used when making a shared object; recompile with -fPIC
/usr/local/lib/libpython3.6m.a: could not read symbols: Bad value
collect2: error: ld returned 1 exit status
CMakeFiles/pydenseflow.dir/build.make:155: recipe for target 'libpydenseflow.so' failed
make[2]: *** [libpydenseflow.so] Error 1
CMakeFiles/Makefile2:215: recipe for target 'CMakeFiles/pydenseflow.dir/all' failed
make[1]: *** [CMakeFiles/pydenseflow.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

repo: open-mmlab/mmaction


text:
Yolov4

hi, I would like to know if Yolov4 has been implemented and where can I find it? Thank you

Yolov4 is not implemented in mmdet, but we will welcome it if you want to support it in our codebase.

source: gharchive/issue

repo: open-mmlab/mmdetection

license_type: permissive

license_source: github-api


text:
RuntimeError: fused_bias_leakyrelu_op_impl: implementation for device cuda:0 not found.

Thanks for your error report and we appreciate it a lot.
Describe the bug
安装配置环境的时候出错，当运行到
fake_imgs = sample_unconditional_model(model, 4)
报错
Reproduction

What command or script did you run?

# 指定 config配置文件路径
config_file = 'configs/styleganv2/stylegan2_c2_lsun-church_256_b4x8_800k.py'

# 指定预训练模型权重文件路径
checkpoint_file = 'https://download.openmmlab.com/mmgen/stylegan2/official_weights/stylegan2-church-config-f-official_20210327_172657-1d42b7d1.pth'

# 初始化generatvie
model = init_model(config_file, checkpoint_file, device='cuda:0')

# 生成图像测试
fake_imgs = sample_unconditional_model(model, 4)

Did you make any modifications on the code or config? Did you understand what you have modified?
没有
What dataset did you use?
教程中自带的

Environment
>>> torch.__version__
'1.11.0+cu113'

>>> torchvision.__version__
'0.12.0+cu113'

>>> torchaudio.__version__
'0.11.0+cu113'

>>> mmcv.__version__
'1.7.0'

Please run python mmgen/utils/collect_env.py to collect necessary environment information and paste it here.

/home/load/.local/lib/python3.10/site-packages/mmcv/__init__.py:20: UserWarning: On January 1, 2023, MMCV will release v2.0.0, in which it will remove components related to the training process and add a data transformation module. In addition, it will rename the package names mmcv to mmcv-lite and mmcv-full to mmcv. See https://github.com/open-mmlab/mmcv/blob/master/docs/en/compatibility.md for more details.
  warnings.warn(
sys.platform: linux
Python: 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]
CUDA available: True
CUDA_HOME: None
GPU 0: NVIDIA GeForce RTX 3060 Laptop GPU
GCC: gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
PyTorch: 1.11.0+cu113
PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.5.2 (Git Hash a9302535553c73243c632ad3c4c80beec3d19a1e)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.3
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
  - CuDNN 8.2
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.11.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,

TorchVision: 0.12.0+cu113
OpenCV: 4.6.0
MMCV: 1.7.0
MMGen: 0.7.2+d1039e0
MMCV Compiler: GCC 11.3
MMCV CUDA Compiler: not available

You may add addition that may be helpful for locating the problem, such as

How you installed PyTorch [e.g., pip, conda, source]
Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)
echo $PATH
/home/load/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin

Error traceback
If applicable, paste the error trackback here.
Traceback (most recent call last):
  File "/home/load/exe.py", line 24, in <module>
    fake_imgs = sample_unconditional_model(model, 4)
  File "/home/load/.local/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/load/mmgeneration/mmgen/apis/inference.py", line 80, in sample_unconditional_model
    res = model.sample_from_noise(
  File "/home/load/mmgeneration/mmgen/models/gans/base_gan.py", line 167, in sample_from_noise
    outputs = _model(noise, num_batches=num_batches, **kwargs)
  File "/home/load/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/load/mmgeneration/mmgen/core/runners/fp16_utils.py", line 142, in new_func
    return old_func(*args, **kwargs)
  File "/home/load/mmgeneration/mmgen/models/architectures/stylegan/generator_discriminator_v2.py", line 372, in forward
    styles = [self.style_mapping(s) for s in styles]
  File "/home/load/mmgeneration/mmgen/models/architectures/stylegan/generator_discriminator_v2.py", line 372, in <listcomp>
    styles = [self.style_mapping(s) for s in styles]
  File "/home/load/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/load/.local/lib/python3.10/site-packages/torch/nn/modules/container.py", line 141, in forward
    input = module(input)
  File "/home/load/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/load/mmgeneration/mmgen/models/architectures/stylegan/modules/styleganv2_modules.py", line 108, in forward
    x = self.activate(x, self.bias * self.lr_mul)
  File "/home/load/.local/lib/python3.10/site-packages/mmcv/ops/fused_bias_leakyrelu.py", line 264, in fused_bias_leakyrelu
    return FusedBiasLeakyReLUFunction.apply(input, bias.to(input.dtype),
  File "/home/load/.local/lib/python3.10/site-packages/mmcv/ops/fused_bias_leakyrelu.py", line 169, in forward
    out = ext_module.fused_bias_leakyrelu(
RuntimeError: fused_bias_leakyrelu_op_impl: implementation for device cuda:0 not found.

Hi @xxLoadxx, it seems like mmcv-full was not installed successfully. Please refer to this issue.

repo: open-mmlab/mmgeneration


text:
Rust crate needs fields: description, license. To publish without errors

error: api errors: missing or empty metadata fields: description, license. Please see https://doc.rust-lang.org/cargo/reference/manifest.html for how to upload metadata

:tada: This issue has been resolved in version 1.8.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

authors: BelfordZ shanejonas

repo: open-rpc/generator-client


text:
fix: make add button hidden until needed

Add filters button used to be disabled when not needed. This was confusing to users so now it is hidden until it is needed.
fixes #62

:tada: This PR is included in version 1.1.1 :tada:
The release is available on:

GitHub release
1.1.1
npm package (@latest dist-tag)

Your semantic-release bot :package::rocket:

source: gharchive/pull-request

authors: mattbford openrpc-bastion

repo: open-rpc/logs-react


text:
Missing lodash dep

jade-signer-rpc|feat/openrpc-integration-tests⚡ ⇒ open-rpc-test-coverage -s ./openrpc.json --transport=http --reporter=console
internal/modules/cjs/loader.js:628
    throw err;
    ^

Error: Cannot find module 'lodash'
Require stack:
- /usr/local/lib/node_modules/@open-rpc/test-coverage/node_modules/@open-rpc/schema-utils-js/build/src/generate-method-id.js
- /usr/local/lib/node_modules/@open-rpc/test-coverage/node_modules/@open-rpc/schema-utils-js/build/src/index.js
- /usr/local/lib/node_modules/@open-rpc/test-coverage/bin/cli.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:625:15)
    at Function.Module._load (internal/modules/cjs/loader.js:527:27)
    at Module.require (internal/modules/cjs/loader.js:683:19)
    at require (internal/modules/cjs/helpers.js:16:16)
    at Object.<anonymous> (/usr/local/lib/node_modules/@open-rpc/test-coverage/node_modules/@open-rpc/schema-utils-js/build/src/generate-method-id.js:3:16)
    at Module._compile (internal/modules/cjs/loader.js:776:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
    at Module.load (internal/modules/cjs/loader.js:643:32)
    at Function.Module._load (internal/modules/cjs/loader.js:556:12)
    at Module.require (internal/modules/cjs/loader.js:683:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/usr/local/lib/node_modules/@open-rpc/test-coverage/node_modules/@open-rpc/schema-utils-js/build/src/generate-method-id.js',
    '/usr/local/lib/node_modules/@open-rpc/test-coverage/node_modules/@open-rpc/schema-utils-js/build/src/index.js',
    '/usr/local/lib/node_modules/@open-rpc/test-coverage/bin/cli.js'
  ]
}

:tada: This issue has been resolved in version 1.0.8 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

authors: mersinvald openrpc-bastion

repo: open-rpc/test-coverage


text:
fix: update schema utils

fixes #63

:tada: This PR is included in version 1.0.8 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

authors: openrpc-bastion shanejonas


text:
Feature: Add Svelete in the list of available languages to filter on

Type of feature
🍕 Feature
Current behavior
Svelte is not a filter provided.
Suggested solution
Add svelte to the dropdown. I will attempt to do this myself.
Additional context
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Contributing Docs

[X] I agree to follow this project's Contribution Docs

Thanks for the issue @brittneypostma! Go for it! The changes will need to be made in https://github.com/open-sauced/app/blob/1073b1d861acf01cb200323f0b1f36e2bcf86f9e/lib/utils/getInterestOptions.ts .

authors: brittneypostma nickytonline

repo: open-sauced/app


text:
Feature: Support personal workspaces

As a user of the application, I want to have a personal workspace to house my contributor insights and contributor repository insights where the landing page is like the current insights hub.

[x] #2652
[ ] #2653
[ ] #2654
[ ] #2655
[x] https://github.com/open-sauced/api/issues/510

@nickytonline and @isabensusan wanted to see if we can track private workspaces here too.
As a user, I should have an option to switch my workspace to private (with paid upgrade).

After https://github.com/open-sauced/api/issues/575 is merged in workspaces that are public should just work. We should modify the settings page to only be in the tabs if you are an owner though and 404 on that page if someone is not the owner and the workspace is public. The API guards against this, but it'd be nice to just not allow non-owners to navigate to the settings page of a workspace.

authors: bdougie nickytonline


text:
Feature: Add end to end testing framework

Type of feature
✅ Test
Current behavior
There are currently zero end to end test in this app and they are needed to start capturing some of the issues in the app.
Suggested solution
Can we explore the following and make a decision on the best framework moving forward?

https://www.cypress.io/
https://playwright.dev/
or combination of both

Example tests to ship.

As a user, I should be able to log in
As a user, I should not be able to vote with out being logged in.

Additional context
This issue should be only be adding the test framework and one test. Please do not add the whole test suite.
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Contributing Docs

[X] I agree to follow this project's Contribution Docs

I’d give playwright a +1 for what that’s worth. I’ve done some stuff with it in my day job and I like it so far. Also I think the newer versions might have some good stuff with respect to being better with react apps (getting “testing library” style locators). https://playwright.dev/docs/release-notes#version-127

authors: bdougie mtfoley

repo: open-sauced/hot

license: MIT


text:
Review and enhance tracing support for ad service service (Java)

The following is a list of requirements that we need to evaluate before declaring v1 for trace telemetry. These requirements are across the entire application; Not all services will meet all requirements. Determine the relevant features for this service.

[x] Automatic Instrumentation is being used where appropriate.
[ ] Library instrumentation is being used if automatic instrumentation is unavailable.
[x] Services extend automatic instrumentation.

[x] New attributes/events attached to existing spans.
[x] New spans are being created from existing spans.

[ ] Automatic and Manual Context Propagation are being demonstrated.
[ ] Telemetry is not being sampled upfront.
[ ] Telemetry is not being filtered upfront.
[ ] Baggage is being set and read appropriately (i.e., baggage must be explicitly set as attributes on spans)
[ ] Update the centralized tracing doc

Referencing: https://github.com/open-telemetry/opentelemetry-demo-webstore/issues/42
Dependent on this item: https://github.com/open-telemetry/opentelemetry-demo-webstore/issues/40

Tracing is code complete

repo: open-telemetry/opentelemetry-demo


text:
[multimod] change return code for diff

This matches the behaviour it was intended on replacing in the collector repository. It returns an error if there's no diff to ensure the build does not proceed with the release in that case.
This fixes https://github.com/open-telemetry/opentelemetry-collector/issues/8149

Pinging @mx-psi for a review :)

Thanks for the reviews @pellared & @mx-psi. This functionality change always felt a bit strange to me, as the return for diff is the opposite of other diff tools. I've decided to close the PR and change the logic in the collector's build pipeline to accommodate for the behaviour that's consistent with other tools.


text:
chore(deps): lock file maintenance

Supersedes #2225 - has some changes to it though:

cucumber did not have node types pinned, so the build failed
nestjs instrumentation was missing a dependency that was required for tests to run properly

Nice work.

Some other versions of @types/node that I see.
├─┬ @opentelemetry/instrumentation-fastify@0.43.0 -> ./plugins/node/opentelemetry-instrumentation-fastify
│ └── @types/node@18.15.3
├─┬ @opentelemetry/propagator-aws-xray-lambda@0.53.0 -> ./propagators/propagator-aws-xray-lambda
│ └── @types/node@18.6.5
├─┬ @opentelemetry/propagator-aws-xray@1.26.0 -> ./propagators/propagator-aws-xray
│ └── @types/node@18.6.5 deduped

Those are older than the "blessed" version "18.18.14". Perhaps that's fine then.

nestjs instrumentation was missing a dependency that was required for tests to run properly

Hrm, perhaps I broke that in this PR ~7w ago then: https://github.com/open-telemetry/opentelemetry-js-contrib/pull/2464/files#diff-5bd19fcb42c8adf472fa8ddceb76b947ab770bbb899a1cacc40dbf4aaa138531L2

Something is up wth the runner for TAV for PR / tav / Run test-all-versions (22) (pull_request). It has been almost 2h running so far.

authors: pichlermarc trentm

repo: open-telemetry/opentelemetry-js-contrib


text:
feat: user config + default config merge for MeterProvider, Meter

Please review https://github.com/open-telemetry/opentelemetry-js/pull/1493 first, it is simpler than this PR, but shares some similarities.
Which problem is this PR solving?

partially #1417

Short description of the changes
@naseemkullah
Member
naseemkullah commented on Sep 3 •
Which problem is this PR solving?
partially #1417
Short description of the changes

Deep merge user supplied Meter and MeterProvider configs into default config by means of lodash.merge

Please review #1493 first, it is simpler than this PR, but shares some similarities.

lint failing in untouched file /home/runner/work/opentelemetry-js/opentelemetry-js/packages/opentelemetry-web/src/utils.ts
/home/runner/work/opentelemetry-js/opentelemetry-js/packages/opentelemetry-web/src/utils.ts
  128:70  error  Replace `⏎····PerformanceResourceTiming⏎··` with `PerformanceResourceTiming`  prettier/prettier

✖ 1 problem (1 error, 0 warnings)
  1 error and 0 warnings potentially fixable with the `--fix` option.

repo: open-telemetry/opentelemetry-js


text:
eslint configuration for getting-started examples

fixes #1390
Which problem is this PR solving?

Adds linting to getting-started directory.

Short description of the changes

I tried to use the similar configuration as examples directory. Although the ts-example needed another configuration.
I have added two lint scripts to the pacakge.json but they are not automatically being executed before the commit. This is the case for examples as well.

Seems like the test is flaky. How can I retrigger the tests?

eslint disallows strict mode directives if the sourceType is module (Docs]).

The examples are not ecmascript modules so the sourceType for at least the examples should be script

Is there any way this can use the already existing eslint rules we have? Don't want to have to maintain two different sets of rules.

Is there any way this can use the already existing eslint rules we have?

I changed the eslintrc to import the eslintrc from the examples directory.

authors: alisabzevari dyladan


text:
Add Zipkin exporter factory

Add the Zipkin exporter configuration factory using the new configuration format. This does not bring many of the settings from the old configuration since they won't be used. In a sub-sequent PR the code of the exporter itself will be changed.
Closes #34

Codecov Report

Merging #207 into master will increase coverage by 0.02%.
The diff coverage is 82.6%.

@@            Coverage Diff             @@
##           master     #207      +/-   ##
==========================================
+ Coverage   72.74%   72.77%   +0.02%
==========================================
  Files         107      108       +1
  Lines        6108     6122      +14
==========================================
+ Hits         4443     4455      +12
- Misses       1438     1440       +2
  Partials      227      227

Impacted Files
Coverage Δ

defaults/defaults.go
78.57% <100%> (+0.79%)
:arrow_up:

exporter/zipkinexporter/zipkin.go
74.01% <100%> (+0.38%)
:arrow_up:

exporter/zipkinexporter/factory.go
77.77% <77.77%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e87131c...e1797e1. Read the comment docs.

@ccaraman I will do a separate PR for changes in the exporters README and will include the changes related to Zipkin.

Removed.

authors: codecov-io pjanotti

repo: open-telemetry/opentelemetry-service


text:
[CI] Textlint: properly check for Java capitalization & more

Contributes to #3458
Followup and alternative (better) solution to #3492
Adds a proper patter to terminology to recognize and fix "Java" mis-capitalizations. In particular, it avoids reporting false positives:

File names: file.java
Repo names: opentelemetry-java
IDs: java.util.List, as they might appear, e.g., in markdown link-text

Drops allow pattern
Drops "Java" word from terminology list
Makes fixes to doc pages
Switches to interpreting .yml files a markdown. This gives us better checking, with fewer false positives, especially for fields that contain markdown

Preview: e.g.,

https://deploy-preview-3497--opentelemetry.netlify.app/docs/instrumentation/java/manual/

(A few more fixes are needed here, for other related issues. I'll submit those after my next meeting. Setting to Draft status for now.)


text:
"Header Checksum" in ipv4 header - what is the relevance of pattern here ?

Should not the values (enum) 'valid checksum' and 'invalid checksum' suffice instead of pattern ?

@ajbalogh we should probably get rid of the pattern for any fields that require checksum. Reasoning is, checksums are usually calculated and embedded right before egress. So whatever we put there might be overwritten.
Although, we might want to allow user to state whether they want to embed bad checksum or good checksum.

Agree.  @ajbalogh thoughts?

@indraniBh @ashutshkumr do you guys want to make a proposal on what the model would look like?  (what would go in place of the pattern concept, that can be multi-purposed to various checksum/crc needs for potentially different packet headers that have those type of needs)?

We do not have a proposal, just wanted to highlight the problem. Most probably we'll need a checksum container to be assigned to all possible checksum fields in l2-4.
e.g.
# automatically insert correct checksum
h = FlowHeader(choice='ipv4', ipv4=Ipv4())
h = FlowHeader(choice='ipv4', ipv4=Ipv4(header_checksum=Checksum(choice='good')))

# automatically insert incorrect checksum
h = FlowHeader(choice='ipv4', ipv4=Ipv4(header_checksum=Checksum(choice='bad')))

# 16-bit checksum
h = FlowHeader(choice='ipv4', ipv4=Ipv4(header_checksum=Checksum(choice='custom', custom='aabb')))
# 32-bit checksum
h = FlowHeader(choice='ethernet', ethernet=Ethernet(fcs=Checksum(choice='custom', custom='aabbccdd')))

NOTE: We can only provide fixed value for custom checksum (ask traffic engine to insert specific checksum before egress).

I don't think it needs a different container as its just another packet field that can be standard or non-standard.
As a user if I'm not interested in the checksum I'm letting the system do it for me - the system's version of a checksum.
If I'm interested in setting the checksum I would like to set it to a single value, value list, counter etc. which will result in either a good or bad checksum.

If one wants to insert bad checksum, they shouldn't rely on providing custom checksum (which requires making sure it's incorrect).
Because, we cannot guarantee that packet contents won't be altered (most probable cause is embedding instrumentation).
Hence, it's almost always easier to just request for bad checksum insertion, which we cannot do with Pattern.
As for custom checksum insertion, I don't have anything against multiple values. If it's not supported by traffic engine, we can always raise it as error.

x-field-pattern extension allows for specifying format: checksum which generates a choice defaulted to an auto generated system checksum with the option for a user to specify their own custom checksum.  Available in release 0.3.1

authors: ajbalogh ankur-sheth ashutshkumr indraniBh winstonliu1111

repo: open-traffic-generator/models


text:
fix(building-rollup): remove legacy scripts from modern watch mode

Found an error when using the rollup modern setup in watch mode.
every time you change a file, there is be a bunch of systemjs imports at the bottom of the page, which should only have been used for the legacy setup.

Seems to be a straightforward logical if else error:
https://github.com/open-wc/open-wc/blob/a064acb30c21fac2b79911b001aa49a0362c41a7/packages/building-rollup/plugins/rollup-plugin-modern-web/rollup-plugin-modern-web.js#L243-L251
that should be solved by:
else if (pluginConfig.legacy && !writtenLegacyModules) {

To reproduce:

use this standard modern rollup config:

const createDefaultConfig = require('@open-wc/building-rollup/modern-config');
const config = createDefaultConfig({
  input: './index.html'
});

run rollup in watch mode
inspect the index file in the dist folder
save a file
inspect the index file in the dist folder

Thanks !

authors: LarsDenBakker kasperstorgaard

repo: open-wc/open-wc


text:
Openid support in 0.3.6

Bug Report
I tested the new open-webui:0.3.6 with openid support.
Docu: https://docs.openwebui.com/tutorial/sso/
Docker Compose FIle:
environment:
- OAUTH_CLIENT_ID=xxxxxxxxxxx
- OAUTH_CLIENT_SECRET=xxxxxxxxxxxx
- OPENID_PROVIDER_URL=https://domain/auth/realms/""Relam""/.well-known/openid-configuration
- OAUTH_PROVIDER_NAME=Keycloak
- OAUTH_SCOPES=
Get the error >>> KeyError: 'userinfo'   UserInfo = token["userinfo"]
I guess I do something wrong by the scope.
Environment

Open WebUI Version: 0.3.6

Logs and Screenshots
NFO:     89.14.165.50:0 - "GET /oauth/oidc/callback?state=tJRmNbMLY0LMuaMc9hFlpaxz4TaV2P&session_state=f60c6e62-4170-4a05-8fc9-15df062542f2&code=7c2e0aa9-0680-4cdb-8536-b266e0d3415d.f60c6e62-4170-4a05-8fc9-15df062542f2.5f683213-517f-4858-acd9-6086beee869e HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
  + Exception Group Traceback (most recent call last):
  |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 87, in collapse_excgroups
  |     yield
  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 190, in call
  |     async with anyio.create_task_group() as task_group:
  |   File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 680, in aexit
  |     raise BaseExceptionGroup(
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 435, in run_asgi
    |     result = await app(  # type: ignore[func-returns-value]
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in call
    |     return await self.app(scope, receive, send)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in call
    |     await super().call(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in call
    |     await self.middleware_stack(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in call
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in call
    |     await self.app(scope, receive, _send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/sessions.py", line 85, in call
    |     await self.app(scope, receive, send_wrapper)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in call
    |     with collapse_excgroups():
    |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in exit
    |     self.gen.throw(typ, value, traceback)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in call
    |     response = await self.dispatch_func(request, call_next)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/app/backend/main.py", line 804, in update_embedding_function
    |     response = await call_next(request)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
    |     raise app_exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    |     await self.app(scope, receive_or_disconnect, send_no_error)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in call
    |     with collapse_excgroups():
    |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in exit
    |     self.gen.throw(typ, value, traceback)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in call
    |     response = await self.dispatch_func(request, call_next)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/app/backend/main.py", line 795, in check_url
    |     response = await call_next(request)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
    |     raise app_exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    |     await self.app(scope, receive_or_disconnect, send_no_error)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in call
    |     await self.app(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in call
    |     with collapse_excgroups():
    |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in exit
    |     self.gen.throw(typ, value, traceback)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in call
    |     response = await self.dispatch_func(request, call_next)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/app/backend/main.py", line 768, in dispatch
    |     response = await call_next(request)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
    |     raise app_exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    |     await self.app(scope, receive_or_disconnect, send_no_error)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in call
    |     with collapse_excgroups():
    |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in exit
    |     self.gen.throw(typ, value, traceback)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in call
    |     response = await self.dispatch_func(request, call_next)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/app/backend/main.py", line 624, in dispatch
    |     response = await call_next(request)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
    |     raise app_exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    |     await self.app(scope, receive_or_disconnect, send_no_error)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in call
    |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    |     await app(scope, receive, sender)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 756, in call
    |     await self.middleware_stack(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 776, in app
    |     await route.handle(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 297, in handle
    |     await self.app(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 77, in app
    |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    |     await app(scope, receive, sender)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 72, in app
    |     response = await func(request)
    |                ^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
    |     raw_response = await run_endpoint_function(
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    |     return await dependant.call(**values)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/app/backend/main.py", line 1894, in oauth_callback
    |     user_data: UserInfo = token["userinfo"]
    |                           ~~~~~^^^^^^^^^^^^
    | KeyError: 'userinfo'
    +------------------------------------
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
Browser Console Logs:
Internal Server Error
Docker Container Logs:
[Include relevant Docker container logs, if applicable]

I solved the problem. I forgot to add  ENABLE_OAUTH_SIGNUP=true
Sorry my mistake.
environment:

ENABLE_OAUTH_SIGNUP=true
OAUTH_CLIENT_ID=xxxxxxxxxxx
OAUTH_CLIENT_SECRET=xxxxxxxxxxxx
OPENID_PROVIDER_URL=https://domain/auth/realms/""Relam""/.well-known/openid-configuration
OAUTH_PROVIDER_NAME=Keycloak
OAUTH_SCOPES=

@T-Herrmann-WI  HI，Are you using a keycloak? Can you refer to your docker-compose?

From the Keycloak or Ollama Stack?

Could you share us how to implement Keycloak?

repo: open-webui/open-webui


text:
VPS Install

Hello,
I didn't find anywhere a HOWTO for installing on VPS Like Virtualmin.
Please some help.

Virtualmin is not a VPS, is a dashboard to manage VPS.
📝 There's a link to manual install instructions in https://www.ollama.com/download .

You want to say that Ollama is better LLM Management tool from Open WebUI?

authors: duhow team-boo


text:
i18n: added hungarian translation hu-HU

Pull Request Checklist
Note to first-time contributors: Please open a discussion post in Discussions and describe your changes before submitting a pull request.
Before submitting, make sure you've checked the following:

[x] Target branch: Please verify that the pull request targets the dev branch.
[x] Description: Provide a concise description of the changes made in this pull request.
[x] Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
[x] Documentation: Have you updated relevant documentation Open WebUI Docs, or other documentation sources?
[x] Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?
[x] Testing: Have you written and run sufficient tests for validating the changes?
[x] Code review: Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
[x] Prefix: To cleary categorize this pull request, prefix the pull request title, using one of the following:

BREAKING CHANGE: Significant changes that may affect compatibility
build: Changes that affect the build system or external dependencies
ci: Changes to our continuous integration processes or workflows
chore: Refactor, cleanup, or other non-functional code changes
docs: Documentation update or addition
feat: Introduces a new feature or enhancement to the codebase
fix: Bug fix or error correction
i18n: Internationalization or localization changes
perf: Performance improvement
refactor: Code restructuring for better maintainability, readability, or scalability
style: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc.)
test: Adding missing tests or correcting existing tests
WIP: Work in progress, a temporary label for incomplete or ongoing work

Changelog Entry

i18n: added hungarian translation hu-HU

Description

[Concisely describe the changes made in this pull request, including any relevant motivation and impact (e.g., fixing a bug, adding a feature, or improving performance)]

Added

[List any new features, functionalities, or additions]

Changed

[List any changes, updates, refactorings, or optimizations]

Deprecated

[List any deprecated functionality or features that have been removed]

Removed

[List any removed features, files, or functionalities]

Fixed

[List any fixes, corrections, or bug fixes]

Security

[List any new or updated security-related changes, including vulnerability fixes]

Breaking Changes

BREAKING CHANGE: [List any breaking changes affecting compatibility or functionality]

Additional Information

[Insert any additional context, notes, or explanations for the changes]

[Reference any related issues, commits, or other relevant information]

Screenshots or Videos

[Attach any relevant screenshots or videos demonstrating the changes]

Thanks!

authors: duroakos tjbck


text:
minLength:6 at operation-name

Decision on the Application Implementation call on 13th or May:
minLength:6 to be added to all operation-name attributes in the ingress bodies.

/v1/relay-operation-update: is only requestBody that has operation-name on the ingress.
Both already have minLength: 6 as a stereotype setting.
No change required.

repo: openBackhaul/RegistryOffice


text:
added city of Kharkiv, Ukraine

Fixes #2176

Hi,
In my test last week there was an issue with Kharkiv. Basically the coordinate system it seems to be following is wrong and in my tests Kharkiv ended up deep inside Kenya. Could you check if this happens for you too? I am pretty sure it is using one of the Pulkovo projections, but the offsets from origin are clearly wrong.
Sergiy

Records still appear to be landing in Kenya.  We'll investigate.

Yeah that preview map should be a tip-off.

Oh yeah, good point.  I see a lot of dots and assume that it's correct.

authors: migurski sergiyprotsiv trescube

repo: openaddresses/openaddresses


text:
Crash on import

I'm trying to run some example code in pybullet, see https://github.com/bulletphysics/bullet3/issues/1234#issuecomment-316521093, that is using baselines but I'm getting an error on import, and they mentioned this is most likely an upstream issue.
I'm on python 2.7.13 on OS X. Perhaps this is a problem with baselines?
athundt at Andrews-2013-MacBook-Pro-2 in ~/src/bullet3/examples/pybullet/gym on master!
± python train_pybullet_racecar.py
pybullet build time: Jul 17 2017 18:59:54
Couldn't import dot_parser, loading of dot files will not be possible.
Traceback (most recent call last):
  File "train_pybullet_racecar.py", line 4, in <module>
    from baselines import deepq
  File "/usr/local/lib/python2.7/site-packages/baselines/deepq/__init__.py", line 2, in <module>
    from baselines.deepq.build_graph import build_act, build_train  # noqa
  File "/usr/local/lib/python2.7/site-packages/baselines/deepq/build_graph.py", line 71, in <module>
    import baselines.common.tf_util as U
  File "/usr/local/lib/python2.7/site-packages/baselines/common/tf_util.py", line 3, in <module>
    import builtins
ImportError: No module named builtins

train_kuka_grasping.py
± python train_kuka_grasping.py
pybullet build time: Jul 17 2017 18:59:54
Couldn't import dot_parser, loading of dot files will not be possible.
Traceback (most recent call last):
  File "train_kuka_grasping.py", line 4, in <module>
    from baselines import deepq
  File "/usr/local/lib/python2.7/site-packages/baselines/deepq/__init__.py", line 2, in <module>
    from baselines.deepq.build_graph import build_act, build_train  # noqa
  File "/usr/local/lib/python2.7/site-packages/baselines/deepq/build_graph.py", line 71, in <module>
    import baselines.common.tf_util as U
  File "/usr/local/lib/python2.7/site-packages/baselines/common/tf_util.py", line 3, in <module>
    import builtins
ImportError: No module named builtins

tf version:
± python -c 'import tensorflow as tf; print(tf.__version__)'
1.2.0

I installed by running pip install baselines, and a full list of installed packages is at https://github.com/bulletphysics/bullet3/issues/1234#issuecomment-316520945

Python 2.7 is not supported. Please use Python 3.0

Easier said than done...

@siemanko Also perhaps the python2 pip package should then be removed or somehow disabled, I installed the package via pip and the install succeeded.

Heck, I've been using my personal utility library for days and mid-day today with no change the mere import of openai in python crashes not just the python script, but the entire calling process.
I changed nothing.

And now it is working again. Go figure
On Tue, Jun 6, 2023 at 10:35 PM Claire CJS @.***> wrote:

Heck, I've been using my personal utility library for days and mid-day
today with no change the mere import of openai in python crashes not just
the python script, but the entire calling process.
I changed nothing.
—
Reply to this email directly, view it on GitHub
https://github.com/openai/baselines/issues/57#issuecomment-1579775769,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAGAYV4I5OU2RJNEVOBLDBLXJ7SFLANCNFSM4DTVLS6A
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
Cheers,
-Claire

authors: ClaireCJS ahundt siemanko

repo: openai/baselines

license: mit

license_source: bigquery


text:
Refactor Customer class to use variable for URL generation

In this commit, the Customer class was refactored to use a variable for URL generation, which improves code clarity and separates the URL generation logic from the API request logic. These changes make the Customer class more explicit and user-friendly, and will facilitate future modifications to the code. Overall, this update improves the usability of the Customer class for open source projects.

Hi @MuleHakim, thanks for the PR!
While code styles are largely a matter of preference, I'm going to argue to that the code, as written, is sufficiently readable given the complexity of the file. Happy to have a larger discussion about this.

authors: MuleHakim hallacy

repo: openai/openai-python


text:
代码语法不支持 3.8 以下版本 Python

是的，被你发现了。昨天做了一下紧急处理，因为requests升级后导致USBClient类不能用了。但是改的比较暴力，用了一些3.8+才能支持的语法。

我在想要不要，只支持3.8以上好了。

authors: codeskyblue youngfreeFJS

repo: openatx/facebook-wda


text:
Enable running the 'Update translations' workflow manually

This allows updating the translations manually just before cutting a release.

I would also prefer to prevent contributors from editing translations outside of crowdin. This would cut down the number of possible merge conflicts on open PRs.

Sorry, I fat-fingered the wrong button.

opencast/opencast has a pretty similar workflow. See here for details: https://github.com/opencast/opencast/blob/develop/.github/workflows/update-translations.yml

opencast/opencast has a pretty similar workflow. See here for details: https://github.com/opencast/opencast/blob/develop/.github/workflows/update-translations.yml

Right, so what I was wondering is, since that workflow also does not have workflow_dispatch, how do the Opencast release managers ensure an up-to-date translation when cutting releases. If it's better, we could adopt it, if it's worse, they could adopt this.

The Opencast Release Manager Guide leaves you a bit in the dark about this. It's not overly specific...

IIRC, Crowdin pushes changes nightly, or at least regularly.  The assumption on the RM side has been that the translations are current enough to cut a release at any point, and fixes would appear in later point releases anyway.

The crowdin updates are updated via a cron-ed weekly GHA workflow.

authors: Arnei gregorydlogan ziegenberg

repo: opencast/opencast-editor


text:
Tweet.java:39-40: Pull out the system proerties, same...

The puzzle 241-12bd8c89 (from #241) in src/main/java/com/amihaiemil/charles/github/Tweet.java (lines 39-40) has to be resolved: "Pull out the system proerties, same as was done in the aws layer, to simplify the unit tests."
The puzzle was created by amihaiemil on 17-Aug-17.
Estimate: 30 minutes, role: IMP.
If you have any technical questions, don't ask me, submit new tickets instead. The task will be "done" when the problem is fixed and the text of the puzzle is removed from the source code. Here is more about PDD and about me.

@charlesmike index this new page please

@charlesmike index this new page please

@amihaiemil thank you for the command, the indexing process started just now. I will let you know of the outcome shortly. In the meantime, you can check the logs here.

@charlesmike index this new page please

@amihaiemil Index command executed successfully!
Add the widget script to the website in order to make the search functionality available.
See logs for more details.
If you notice any inexplicable errors, please open an issue here

The puzzle 241-12bd8c89 has disappeared from the source code, that's why I closed this issue.

authors: 0pdd amihaiemil charlesmike

repo: opencharles/charles-rest


text:
Compare with original OpenSCM file

Something seem to have gone missing
1,3c1
< Source,https://www.ghgprotocol.org/sites/default/files/ghgp/Global-Warming-Potential-Values%20%28Feb%2016%202016%29_1.pdf,https://www.ghgprotocol.org/sites/default/files/ghgp/Global-Warming-Potential-Values%20%28Feb%2016%202016%29_1.pdf,https://www.ghgprotocol.org/sites/default/files/ghgp/Global-Warming-Potential-Values%20%28Feb%2016%202016%29_1.pdf,"AR5CCFGWP100 are AR5 100 year GWPs with climate carbon cycle feedbacks: https://www.ipcc.ch/site/assets/uploads/2018/07/WGI_AR5.Chap_.8_SM.pdf, Table 8.SM.16 and https://www.ipcc.ch/site/assets/uploads/2018/02/WG1AR5_Chapter08_FINAL.pdf , Table 8.7 (page 714)."
< OpenSCMMetricName,SARGWP100,AR4GWP100,AR5GWP100,AR5CCFGWP100
< OpenSCM species,,,,
---
> Species,SARGWP100,AR4GWP100,AR5GWP100,AR5CCFGWP100
86c84
< HFE365mcf3,,,1,1
---
> HFE365mcf3,,,,1

I was just about to open the same issue...

Thanks for checking! :-)
Should be fixed now (https://github.com/openclimatedata/globalwarmingpotentials/commit/f29a889762d391fdf5dd8b1eb7c35275e8d65f13):
wget https://raw.githubusercontent.com/openscm/openscm-units/master/src/openscm_units/data/metric_conversions.csv

diff metric_conversions.csv globalwarmingpotentials.csv
1,3c1
< Source,https://www.ghgprotocol.org/sites/default/files/ghgp/Global-Warming-Potential-Values%20%28Feb%2016%202016%29_1.pdf,https://www.ghgprotocol.org/sites/default/files/ghgp/Global-Warming-Potential-Values%20%28Feb%2016%202016%29_1.pdf,https://www.ghgprotocol.org/sites/default/files/ghgp/Global-Warming-Potential-Values%20%28Feb%2016%202016%29_1.pdf,"AR5CCFGWP100 are AR5 100 year GWPs with climate carbon cycle feedbacks: https://www.ipcc.ch/site/assets/uploads/2018/07/WGI_AR5.Chap_.8_SM.pdf, Table 8.SM.16 and https://www.ipcc.ch/site/assets/uploads/2018/02/WG1AR5_Chapter08_FINAL.pdf , Table 8.7 (page 714)."
< OpenSCMMetricName,SARGWP100,AR4GWP100,AR5GWP100,AR5CCFGWP100
< OpenSCM species,,,,
---
> Species,SARGWP100,AR4GWP100,AR5GWP100,AR5CCFGWP100

authors: danielhuppmann rgieseke

repo: openclimatedata/globalwarmingpotentials


text:
Add validation on Location object

Add validation on Location object.
Currently the location are in osgb mapping, so should check
x>0, x < need to look up
y>0, y < need to look up

Hi @peterdudfield is this still open, I would like to work on it, thanks.

Yea, this would be amazing

Thanks, will start working on this once I am done with #25.

@peterdudfield I was looking where the Location object is getting instantiated, seems like its done in ocf_datapipes/select/location_picker.py, so should we just continue if the location.x or location.y is < 0 ?
and also, sorry, but couldn't understand what do you mean by "need to look up" in your original comment "x or y < need to look up" ?
Thanks.

I think it would be great to use pydantic (validators)[https://docs.pydantic.dev/latest/usage/validators/]
Ah yea I need to look up the maximum values, this website gives both the maximum and the minimum - https://epsg.io/27700

As a bonus you could put a extra field saying
coordinate_system' which could only be osgborlat_lon`,
And then validating if its lat_lon would be -180 <x< 180 and -90 < y< 90 ( i think)

@peterdudfield sorry wasn't aware of pydantic for validation, good to know about it.
I'll use that and also add another field for coordinate_system
From the website, looks like below are the bounds:
Projected bounds:
-103976.3 -16703.87
652897.98 1199851.44
And based on the Center coordinates of 305371.94    610575.38
looks like
-103976.3 < x < 652897.98
-16703.87 < y < 1199851.44
If my above understanding of the bounds is correct, then looks like x and y both can take values < 0, but in your original message you want to make sure they are > 0?
Could you please explain a little more here, sorry, if I am missing something here.
Thanks.

@peterdudfield sorry wasn't aware of pydantic for validation, good to know about it. I'll use that and also add another field for coordinate_system
From the website, looks like below are the bounds: Projected bounds: -103976.3 -16703.87 652897.98 1199851.44
And based on the Center coordinates of 305371.94 610575.38 looks like -103976.3 < x < 652897.98 -16703.87 < y < 1199851.44
If my above understanding of the bounds is correct, then looks like x and y both can take values < 0, but in your original message you want to make sure they are > 0? Could you please explain a little more here, sorry, if I am missing something here. Thanks.

Yea please use the bounds you have found, not >0

Thanks @peterdudfield
I have made the below updates to the ocf_datapipes/utils/consts.py file:

Added a new optional field named coordinate_system, it defaults to osgb
Added validators for coordinate_system, x and y.
Based on the chosen coordinate_system the min/max values for x and y will be different.

Now for testing:

Since I am not familiar with the entire code base, could you please help check if this looks good?
What test(s) can I run to make sure this will not blow up something else somewhere?
And any other checks we need to make?

class Location(BaseModel):
    """Represent a spatial location."""

    x: float
    y: float
    id: Optional[int]
    coordinate_system: Optional[str] = "osgb"   # ["osgb", "lat_lon"]

    @validator("coordinate_system")
    def validate_coordinate_system(cls, v):
        """Validate 'coordinate_system'"""
        allowed_coordinate_systen = ["osgb", "lat_lon"]
        if v not in allowed_coordinate_systen:
            raise ValueError(f"coordinate_system = {v} is not in {allowed_coordinate_systen}")
        return v

    @validator("x")
    def validate_x(cls, v):
        """Validate 'x'"""
        min_x: float
        max_x: float
        if coordinate_system == "osgb":
            min_x, max_x = -103976.3, 652897.98
        if coordinate_system == "lat_lon":
            min_x, max_x = -180, 180
        if v < min_x or v > max_x:
            raise ValueError(f"x = {v} must be within {[min_x, max_x]} for {coordinate_system} coordinate system")
        return v

    @validator("y")
    def validate_x(cls, v):
        """Validate 'y'"""
        min_y: float
        max_y: float
        if coordinate_system == "osgb":
            min_y, max_y = -16703.87, 1199851.44
        if coordinate_system == "lat_lon":
            min_y, max_y = -90, 90
        if v < min_y or v > max_y:
            raise ValueError(f"y = {v} must be within {[min_y, max_y]} for {coordinate_system} coordinate system")
        return v

Thanks!

Yea that looks really good. Thanks for doing this
Test can be just to

make a validate osgb object
make a validate lat/lon object
make a invalid osgb x object and see if you get an error
make a invalid osgb y object and see if you get an error
5 make a invalid lat_lon x object and see if you get an error
make a invalid lat_lon y object and see if you get an error
Make invalid "coordinate_system" and check you get an error

any extras you want

The other CI tests will check that your code doesnt blow things up

Thanks @peterdudfield !

Is there some reference code/tests in the codebase that I can refer to, to write these tests, that way I can follow the same standards.
Also, once I write the tests, do I need to upload the tests as well to the codebase with my PR? If yes, which directory under tests should these go in?

Yes, thanks!
another issue that I keep encountering is, whenever I try to run any tests, even the already existing ones, it gives me this error.

As far as I know dict are not subscriptable, so the error makes sense to me, but knowing this code base is functional this must have worked. So, am I missing something?
The typing module has Dict (upper case "D") in it and that it subscriptable, but then Dict is not imported in this consts.py file.
So, I am wondering how was this code run before without running into this error.
Appreciate your help in helping me understand this, thanks.

What python version are you using? I think those features come in with python >=3.10

I am using 3.8 since the .readthedocs.yml file says so..

understood, no worries, let me give 3.10 a try...

Thanks @rjmcoder

@peterdudfield Python 3.10 worked, thanks.
I have some of the tests working, finishing up the remaining ones.. shooting to submit a PR by sometime next week.

@peterdudfield I have raised a PR #201 with the fixes.
Feel free to let me know if more changes are needed.
Thanks!

authors: peterdudfield rjmcoder


text:
add max_retries check and exception

Description
This PR addresses the issue of unclear error handling when multiple timeouts occur during data fetching. The fetch_data function has been updated to raise a clear and descriptive error message if the maximum number of retries is reached without successfully fetching the data.
Changes:

Added logic to raise a ConnectionError after exhausting retries, with detailed error messages.
Improved logging to track the retry attempts and include status code or error information.
Ensured that if the data fetch ultimately fails, a concise error message is displayed to the user.

Fixes #18
How Has This Been Tested?
To verify the changes, I simulated network timeouts and ensured the following:

The retry logic triggers appropriately.
After max_retries is reached, the error message is logged, and a ConnectionError is raised.
The function behaves as expected when data is fetched successfully.

Tested with:

[x] Timeout simulation (5 retries)
[x] Successful data fetch after retries

If your changes affect data processing, have you plotted any changes? i.e., have you done a quick sanity check?

[x] Yes

Checklist:

[x] My code follows OCF's coding style guidelines
[x] I have performed a self-review of my own code
[x] I have made corresponding changes to the documentation (if necessary)
[x] I have added tests that prove my fix is effective or that my feature works
[x] I have checked my code and corrected any misspellings

@peterdudfield Can you review this?

repo: openclimatefix/ruvnl-consumer-app


text:
Add LR warm up scheduling

Based off https://twitter.com/jack_kelly/status/1411615500371640320?s=19 might help with metnet training, should be easy to add and experiment with. Plan would be to add it to the generic model interface so we can try it on any of the models

Added for MetNet in #48

repo: openclimatefix/satflow


text:
SanitizeHtml: Allow figure with content-type for iframes

For https://github.com/opencollective/opencollective-frontend/pull/6042

@Betree : Thanks for adding this. But something not working correctly before adding this figure for content types? 🤔

@SudharakaP responsiveness. Before that, the height was fixed to 400px so on mobile you ended up with two black stripes above and below the video. Having this content-type allowed us to add this CSS rule which creates a real 16:9 container.

@SudharakaP
Before:

After:

@SudharakaP responsiveness. Before that, the height was fixed to 400px so on mobile you ended up with two black stripes above and below the video. Having this content-type allowed us to add this CSS rule which creates a real 16:9 container.

@Betree : Thanks. Looks wonderful now. 😄

authors: Betree SudharakaP

repo: opencollective/opencollective-api


text:
[Roadmap] 2022 Q1 Sprint 1

At our latest European retreat in Madrid, we identified the following "Strategic Focus" to help us prioritize projects for the first quarter.

https://github.com/opencollective/opencollective/issues/5140
https://github.com/opencollective/opencollective/issues/5141
https://github.com/opencollective/opencollective/issues/5143
https://github.com/opencollective/opencollective/issues/5148

We then prioritized the following Projects:

https://github.com/opencollective/opencollective/issues/5150 (for Open Collective as a Platform)
https://github.com/opencollective/opencollective/issues/5146 (for Expense Flexibility, as a first step for Recurring Expenses)
https://github.com/opencollective/opencollective/issues/5036 (for Community Governance)
https://github.com/opencollective/opencollective/issues/4691 (for Community Growth, Transparency/Trust)
https://github.com/opencollective/opencollective/issues/4689 (for Community Growth, Onboarding)
https://github.com/opencollective/opencollective/issues/4829 (for Community Growth, Onboarding)

As time allows, we'll continue working on the following Papercuts:

https://github.com/opencollective/opencollective/issues/5084

Sprint facilitator: François Hodierne (@znarf)

At our latest European retreat in Madrid, we identified the following "Strategic Focus" to help us prioritize projects for the first quarter.

https://github.com/opencollective/opencollective/issues/5140
https://github.com/opencollective/opencollective/issues/5141
https://github.com/opencollective/opencollective/issues/5143
https://github.com/opencollective/opencollective/issues/5148

We then prioritized the following Projects:

https://github.com/opencollective/opencollective/issues/5150 (for Open Collective as a Platform)
https://github.com/opencollective/opencollective/issues/5146 (for Expense Flexibility, as a first step for Recurring Expenses)
https://github.com/opencollective/opencollective/issues/5036 (for Community Governance)
https://github.com/opencollective/opencollective/issues/4691 (for Community Growth, Transparency/Trust)
https://github.com/opencollective/opencollective/issues/4689 (for Community Growth, Onboarding)
https://github.com/opencollective/opencollective/issues/4829 (for Community Growth, Onboarding)

As time allows, we'll continue working on the following Papercuts:

https://github.com/opencollective/opencollective/issues/5084

Sprint facilitator: François Hodierne (@znarf)

At our latest European retreat in Madrid, we identified the following "Strategic Focus" to help us prioritize projects for the first quarter.

[Strategic Focus] API, Open Collective as a platform #5140
[Strategic Focus] Flexibility, Automation and Programmability of Contributions & Expenses #5141
[Strategic Focus] Community Governance #5143
[Strategic Focus] Community Growth #5148

We then prioritized the following Projects:

[PROJECT] API completeness #5150 (for Open Collective as a Platform)
[PROJECT] Multi-currency expenses #5146 (for Expense Flexibility, as a first step for Recurring Expenses)
[PROJECT] Setting to prevent submitters approving their own expenses #5036 (for Community Governance)
[PROJECT] Show details for Expenses in transactions list #4691 (for Community Growth, Transparency/Trust)
[PROJECT] New opencollective.com homepage #4689 (for Community Growth, Onboarding)
[PROJECT] Merge profiles onboarding #4829 (for Community Growth, Onboarding)

As time allows, we'll continue working on the following Papercuts:

[PROJECT] Events and Projects QA & papercuts #5084

Sprint facilitator: François Hodierne (@znarf)

authors: AdNaN-rK Paingc109 znarf

repo: opencollective/opencollective


text:
New VAT/GST custom field for expenses

User story
As a host Admin, I wanna make sure that people only uses the invoice option to submit services with a legit tax status. In the EU, you need a company/tax number to get paid for a services you gave to a collective.
As a host Admin, I need my accountant to be able to get these tax numbers in the csv export
Best solution for this problem
Add a mandatory VAT/GST custom field (that some host can have, other not depending on the needs) after the user chose the invoice option.
P1 high frequency, high impact

Custom fields are indeed an option, but I would advocate for implementing explicit support for VAT directly in the code - like what we do for contributions.
Related: https://github.com/opencollective/opencollective/issues/4532

@jfdh shall we close this in favour of #4532?

Yes! Created this in a hurry but i am going to elaborate on #4532

Implemented with https://github.com/opencollective/opencollective/issues/4532

authors: BenJam Betree jfdh


text:
RT-3.32 new README.md

This is for "DSCP based traffic steering from Non-default VRF to Default VRF using Policy based VRF selection plus GUE DECAP and ENCAP"

Pull Request Functional Test Report for #3485 / 2431375e87cca1d63e36332eb4245a01bca056ee
No tests identified for validation.
Help

Pull Request Test Coverage Report for Build 11120518135
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 55.268%

Totals

Change from base Build 11119090068:
0.0%

Covered Lines:
1983

Relevant Lines:
3588

💛  - Coveralls

authors: OpenConfigBot coveralls sachendras

repo: openconfig/featureprofiles


text:
questions about slides

[ ] update "Data Intensive Cluster" image/slide correct? @alexbaden @randalburns @kunallillaney @willgray
[ ] update the "Extract Geometries" slide correct? @alexbaden
[ ] update the "Extract Graphs" slide correct? @kunallillaney
[ ] populate "Other EM Datasets" via LIMS @j6k4m8
[ ] add links to appropriate dataviews for "Data with Volumetric Annotations" @alexbaden
[ ] add links to dataviews for "Other Modalities" @alexbaden @j6k4m8
[ ] replace page #'s with dots
[ ] extract graph documentation @kunallillaney
[ ] case study #2: which graph did they actually use? @willgray

@willgray - please paste link to lucidchart in here.

repo: openconnectome/intro


text:
Change merge strategy to "squash merging"

Hi everybody
I already asked for this in the slack channel #opencost.

As I am one of the maintainer of the Argo CD helm repository, we follow what most chart repositories do:

Can you change this as well here?
/cc: @sntxrr @Pokom

I generally don’t use squash merges, but if everyone else does I don’t care enough not to allow them. Anyone feel strongly against them? @dwbrown2 @Pokom @AjayTripathy @toscott any opinions yeah/nay?

I would prefer that we allow it. Squashing vs Merging will usually turn into a divisive topic, so I don't want to push it unless we're all in agreement.
My primary reason for doing a squash commit is that it keeps the history clean on main. However, the challenge because if one of those small commits that got squashed has an issue, we need to revert the entire change if it's squashed.
I'd say let's vote with :+1: and :-1: on the main issue to get consensus.

Updated to match screenshot.

If squash is what the community does I support that!

authors: AjayTripathy Pokom mattray mkilchhofer

repo: opencost/opencost-helm-chart


text:
Outil de récupération automatique des données

Bonjour,
Voici une ébauche de ce qui pourrait être un outil de récupération automatique de données depuis nos sites sources, dans le but de faciliter et de fiabiliser le travail d’agrégation des données. Ce programme récupère la page du site, en extrait les données et génère le fichier YAML correspondant.
Pour le lancer, il vous faut un interpréteur Python 3.7:
python3.7 -m venv venv
source ./venv/bin/activate  # sous bash, peut varier suivant le shell
pip install -r requirements.txt
python -m gatherer

Il créé automatiquement le fichier dans sante-publique-france/ correspondant à la date du jour.
Les limitations actuelles sont:

pas de vérification de la date du jour,
implémenté pour Santé Publique France seulement,
difficile à lancer pour quelqu'un qui n'est pas dev'.

Voilà, j'attends vos critiques, dites moi si ça vaut la peine de continuer à travailler là dessus, où s'il veut mieux s'attaquer à autre chose.

L'objectif est de récupérer les données régionales, pas les données mondiales ? (juste pour être sûr)

Bonjour @tvuillemin,
Vu le nombre de développeur par ici nous en sommes déjà à 3/4 scrappers :)
Le problème de la démarche est que de toute façon il faut lancer le script, aller voir la page pour vérifier s’il y a pas de nouvelles données ou des avertissements, et archiver la page sur web.archive.org.
Le plus gros de notre problématique actuelle c’est la veille des données des ARS, et le manque de contributeurs actifs.
On espère par ailleurs avoir rapidement les données de SPF par un autre moyen que son site Web !

@jdesboeufs OK, on laisse tomber alors ? Je n'ai pas trouvé les autres outils de récupération en parcourant les dépôts, est-ce que tu peux m'indique où il sont ?
Sinon, le fait de vérifier qu'il n'y a pas de nouvelles données et d'archiver sur web.archive.org peut s'automatiser sans problème. Mais effectivement, si SPF fournit les données autrement que dans du HTML, ça va simplifier bien des choses.
Pour éviter de devoir lancer le script, on peut tout à fait imaginer ça dans un Docker ou une VM quelconque, et exposer une API que l'appli web viendrait taper pour récupérer les données. J'y ai pensé, mais je voulais publier un premier brouillon pour en discuter avant de me lancer dans dans trucs potentiellement inutiles.
Bref, hésite pas à me contacter directement sur Slack si tu as besoin de bras. Je ne sais pas comment faire « de la veille de données des ARS », mais je veux bien apprendre.

@ColinMaudry Oui, j'ai limité aux seules données régionales dans ce premier jet.

Cette issue n'est pas faites pour ça.
Mais @jdesboeufs pour les fichiers de l'ARS et la veille, j'en fais déjà quelqu'un mais je peux en prendre plus sans trop de souci.
Tant que je dois pas tous les avoir fait pour le jour J à 0h :)

authors: ColinMaudry jdesboeufs liszto tvuillemin


text:
new to CVAT. drawing polygon

I have loaded a picture.
kept--> Annotation-->Polygon.
pressed N (Captial N)
I am not able to get the option to mark.
I tried left click as mentioned in https://www.youtube.com/watch?v=XTwfXDh4clI
and saw this also...
https://github.com/opencv/cvat/blob/master/cvat/apps/documentation/user_guide.md#annotation-with-polygons
am I missing anything please ..when I press left the image is also moving

@newmluser , please in chrome browser click F12 and see "console". Let's know if you see any errors.

Resolved on gitter

authors: bsekachev newmluser nmanovic

repo: opencv/cvat