text:
Visualizing weights and convolutions

Hi,
Looking at this Torch tutorial:
http://code.madbits.com/wiki/doku.php?id=tutorial_basics
A Spatial Convolution layer is generated, and its weights are visualized,
then Lena's image is forwarded through the layer and the resulting images are visualized.
It would be nice if Keras would support the visualization of both things (aka include the visualization in its tutorials).

In particular, I'd expect to see the different versions of the picture after each of the convolutions have been applied. The following code, when executed in IPython Notebook, will display an image but not the convolutions over it.
from keras.datasets import mnist
from matplotlib.pyplot import imshow

(X_train, y_train), (X_test, y_test) = mnist.load_data()
inpic = X_train[0][0]

%matplotlib inline
imshow(X_train[0][0]) #shows the first image in the mnist dataset.

from keras.layers.convolutional import Convolution2D
l = Convolution2D(32, 1, 3, 3, border_mode='full')
imshow(l.activation(inpic)) #l.activation returns inpic unchanged, so the image is showed again.

I have a notebook based on keras' CNN example on MNIST that has example code to show the convolution and weights :
https://github.com/julienr/ipynb_playground/blob/master/keras/convmnist/keras_cnn_mnist.ipynb
You cannot call l.activation directly, you have to first have Theano compile the expression. For example :
model = Sequential()

model.add(Convolution2D(32, 1, 3, 3, border_mode='full'))
convout1 = Activation('relu')
model.add(convout1)

convout1_f = theano.function([model.get_input(train=False)], convout1.get_output(train=False))

And then you can call convout1_f on an image and get your result.

@julienr cool stuff!

Here's a fully working example based on @julienr 's comment - running this in IPython Notebook yields neat results:
# Imports
import matplotlib.pyplot as plt

import theano

import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers.convolutional import Convolution2D
from keras.layers.core import Activation

# Model
model = Sequential()

model.add(Convolution2D(28, 1, 3, 3, border_mode='full'))
convout1 = Activation('relu')
model.add(convout1)

# Data loading + reshape to 4D
(X_train, y_train), (X_test, y_test) = mnist_dataset = mnist.load_data()
reshaped = X_train.reshape(X_train.shape[0], 1, X_train.shape[1], X_train.shape[2])

from random import randint
img_to_visualize = randint(0, len(X_train) - 1)

# Generate function to visualize first layer
convout1_f = theano.function([model.get_input(train=False)], convout1.get_output(train=False))
convolutions = convout1_f(reshaped[img_to_visualize: img_to_visualize+1])

%matplotlib inline #which is an alias of the next line
#get_ipython().magic(u'matplotlib inline')
imshow = plt.imshow #alias
plt.title("Image used: #%d (digit=%d)" % (img_to_visualize, y_train[img_to_visualize]))
imshow(X_train[img_to_visualize])

plt.title("First convolution:")
imshow(convolutions[0][0])

print "The second dimension tells us how many convolutions do we have: %s (%d convolutions)" % (
    str(convolutions.shape),
    convolutions.shape[1])

for i, convolution in enumerate(convolutions[0]):
    plt.figure()
    plt.title("Convolution %d" % (i))
    plt.imshow(convolution)

@guy4261 when runing the code i am getting this error " Exception: Layer is not connected                and is not an input layer." can you please help ...

@erfannoury I'll have a look. I have keras 0.1.3 on my machine and I see that the latest version is 0.3.0. Can you please run the following cell to tell me your version?
! pip list | grep -i keras

@guy4261 I think I shouldn't have been mentioned. Anyway, I'm using the latest version of Keras (0.3.0).

This is possible with graphs as well, but graph.get_input() does not give a value suitable for theano.function. Instead you need to grab each input's get_input() result and use that instead. E.g.
    midout_fn = theano.function([graph.inputs['input1'].get_input(train=False),
                                 graph.inputs['input2'].get_input(train=False)],
                                midout.get_output(train=False))

where midout is equivalent of convout1 above.

Is there an easy way to do this with the tensorflow backend? or is making a new network with a different outputs

@kmader I think you should be able to replace theano.function with K.function (where K is the backend imported with from keras import backend as K). Haven't tested it though.

@julienr thanks, that is what I ended up doing and it works fine

@kallewoof @julienr @guy4261  ... I am getting following error for Sequential model.
Using keras 1.0.
convout1_f = theano.function([model.get_input()], convout1.get_output(train=False))
Error :
AttributeError: 'Sequential' object has no attribute 'get_input'

Same issue as saj1919.  As soon as I upgraded to Keras 1.0 I started getting the error
'''
AttributeError: 'Sequential' object has no attribute 'get_input'
'''

I think you should use model.inputs instead of model.get_input() :
convout1_f = K.function(model.inputs, [convout1.output])

And later you have to call it with a list of inputs :
C1 = convout1_f([X])

Full working example with keras 1.0.1 :
https://github.com/julienr/ipynb_playground/blob/master/keras/convmnist/keras_cnn_mnist_v1.ipynb

Ok, now I'm getting a new error:
MissingInputError: ("An input of the graph, used to compute DimShuffle{x,x,x,x}(keras_learning_phase), was not provided and not given a value.Use the Theano flag exception_verbosity='high',for more information on this error.", keras_learning_phase)

The problem seems to be dropout.  Dropout needs to know the learning phase because it has different behavior for train time vs test time.  When I get rid of dropout in my model it works and when I add dropout to your (julienr) model before the layer we want output from I start getting the same error.  How do we tell the backend function the learning phase?
Here, btw is how I modified julienr's model to get the error:
model = Sequential()

model.add(Convolution2D(32, 3, 3, border_mode='valid', input_shape=X_train.shape[1:]))
model.add(Dropout(0.01)) # This breaks it
convout1 = Activation('relu')
model.add(convout1)
model.add(Convolution2D(32, 3, 3))

@datamath28 You need to add K.learning_phase to the list of inputs and specify a value (0 to disable) for it. This is a boolean variable that's used by Dropout (and other train-only layers) to determine if they should have an effect.
I've pushed the fix to my notebook. You need to modify how you create the convout1_f function as follow :
# K.learning_phase() is a flag that indicates if the network is in training or
# predict phase. It allow layer (e.g. Dropout) to only be applied during training
inputs = [K.learning_phase()] + model.inputs

_convout1_f = K.function(inputs, [convout1.output])
def convout1_f(X):
    # The [0] is to disable the training phase flag
    return _convout1_f([0] + [X])

Hello,guys!
Now I wanna get the FC layer's feature datas and use it in SVM to be classified for 2 classes.Here is my main code:

from future import print_function
import cPickle
import theano
from sklearn.ensemble import RandomForestClassifier
from sklearn.svm import SVC
from sklearn.preprocessing import MinMaxScaler
from luobindata8_18 import load_data
import random

def svc(traindata,trainlabel,testdata,testlabel):
print("Start training SVM...")
svcClf = SVC(C=1.0,kernel="rbf",cache_size=900)
svcClf.fit(traindata,trainlabel)
pred_testlabel = svcClf.predict(testdata)
num = len(pred_testlabel)
accuracy = len([1 for i in range(num) if testlabel[i]==pred_testlabel[i]])/float(num)
print("cnn-svm Accuracy:",accuracy)

def rf(traindata,trainlabel,testdata,testlabel):
print("Start training Random Forest...")
rfClf = RandomForestClassifier(n_estimators=2,criterion='gini')
rfClf.fit(traindata,trainlabel)
pred_testlabel = rfClf.predict(testdata)
num = len(pred_testlabel)
accuracy = len([1 for i in range(num) if testlabel[i]==pred_testlabel[i]])/float(num)
print("cnn-rf Accuracy:",accuracy)

if name == "main":
#load data
data, label = load_data()
#shuffle the data
index = [i for i in range(len(data))]
random.shuffle(index)
data = data[index]
label = label[index]
(traindata,testdata) = (data[0:900],data[900:])
(trainlabel,testlabel) = (label[0:900],label[900:])
#use origin_model to predict testdata
origin_model = cPickle.load(open("model8.18.pkl","rb"))
#print(origin_model.layers)

pred_testlabel = origin_model.predict_classes(testdata)
#print(pred_testlabel)
num = len(testlabel)
accuracy = len([1 for i in range(num) if testlabel[i]==pred_testlabel[i]])/float(num)
print(" Origin_model Accuracy:",accuracy)
#define theano funtion to get output of FC layer
get_feature=    theano.function([origin_model.layers[0].input],origin_model.layers[13].output,allow_input_downcast=False)
feature = get_feature(data)
#train svm using FC-layer feature
scaler = MinMaxScaler()
feature = scaler.fit_transform(feature)
svc(feature[0:900],label[0:900],feature[900:],label[900:])
rf(feature[0:900],label[0:900],feature[900:],label[900:])

##############################################
However,here is luobindata8_18 which used for load data from my pictures:

import os
from PIL import Image
import numpy as np
def load_data():
data = np.empty((1040,1,60,60),dtype="float32")
label = np.empty((1040,),dtype="int")
imgs = os.listdir("./sampleyingguang8.18")
num = len(imgs)
for i in range(num):
img = Image.open("./sampleyingguang8.18/"+imgs[i])
arr = np.asarray(img,dtype="float32")
data[i,:,:,:] = arr
if i<623:
label[i] = int(0)
else:
label[i] = int(1)
data /= np.max(data)
data -= np.mean(data)
return data,label
########################################
That's all codes.Please neglect the 1040 sample number even though I really know it is too few for CNN but it is not the key making the error.When I run these codes,the error below happened:

MissingInputError: ("An input of the graph, used to compute DimShuffle{x,x,x,x}(keras_learning_phase), was not provided and not given a value.Use the Theano flag exception_verbosity='high',for more information on this error.", keras_learning_phase)

This error happened in :get_feature = theano.function([origin_model.layers[0].input],origin_model.layers[13].output,allow_input_downcast=False)
#############################################
I really don't know the reason as a Keras fish!

I also have tryed a solution below but it doesn't work:
from keras import backend as K
#and then replace get_feature = theano.function([origin_model.layers[0].input],origin_model.layers[13].output,allow_input_downcast=False)  with get_feature = K.function([origin_model.layers[0].input],origin_model.layers[13].output)

Sorry for my poor english and the wrong format of codes when uploading. @julienr @erfannoury @guy4261 @fchollet @ArkAlista

I haven't reminded the file of "model8.18.pkl".Actually it came from codes below:
from future import absolute_import
from future import print_function
from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation, Flatten
from keras.layers.convolutional import Convolution2D, MaxPooling2D
from keras.optimizers import SGD,Adadelta
import keras.regularizers
from sklearn.ensemble import RandomForestClassifier
from sklearn.svm import SVC
from sklearn.preprocessing import MinMaxScaler
import theano
#from keras.utils import np_utils, generic_utils
from six.moves import range
from luobindata8_18 import load_data
import random,cPickle
from keras.callbacks import EarlyStopping
import numpy as np
np.random.seed(1024)
data, label = load_data()
def create_model():
model = Sequential()
model.add(Convolution2D(32, 5, 5, border_mode='valid',input_shape=(1,60,60)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.3))
model.add(Convolution2D(32,3, 3, border_mode='valid'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.3))

model.add(Convolution2D(64,3, 3, border_mode='valid'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.3))

model.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors
model.add(Dense(128))
model.add(Activation('relu'))
model.add(Dropout(0.3))
model.add(Dense(1))
model.add(Activation('sigmoid'))
return model

#############
#开始训练模型
##############
model = create_model()
#sgd = SGD(lr=0.001, decay=1e-8, momentum=0.9, nesterov=True)
#model.compile(loss='binary_crossentropy', optimizer=sgd,metrics=["accuracy"])
sgd = SGD(lr=0.005, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='binary_crossentropy', optimizer=sgd,metrics=["accuracy"])
#adadelta=Adadelta(lr=1.0, rho=0.95, epsilon=1e-09)
#model.compile(loss='mse', optimizer=adadelta,metrics=["accuracy"])
#model.compile(loss="binary_crossentropy", optimizer=adadelta,metrics=["accuracy"])
index = [i for i in range(len(data))]
random.shuffle(index)
data = data[index]
label = label[index]
(X_train,X_val) = (data[0:900],data[900:])
(Y_train,Y_val) = (label[0:900],label[900:])
#使用early stopping返回最佳epoch对应的model
early_stopping = EarlyStopping(monitor='val_loss', patience=1)
#model.fit(X_train, Y_train,callbacks=[early_stopping])
model.fit(X_train, Y_train, batch_size=30,validation_data=(X_val, Y_val),nb_epoch=15,callbacks=[early_stopping])
pred_testlabel=model.predict_classes(X_val)
#print('true test label:')
#print(Y_val)
#print('cnn test label:')
#print(pre_label)
print('calculate stone and kuang accuracy:')
num=len(Y_val)
stonenum=0
for i in Y_val:
if i==0:
stonenum=stonenum+1
kuangnum=num-stonenum
cnnstone=0
for i in range(num):
if Y_val[i]==pred_testlabel[i] and Y_val[i]==0:
cnnstone=cnnstone+1
stoneaccuracy=float(cnnstone)/float(stonenum)
print(stoneaccuracy)
cnnkuang=0
for i in range(num):
if Y_val[i]==pred_testlabel[i] and Y_val[i]==1:
cnnkuang=cnnkuang+1
kuangaccuracy=float(cnnkuang)/float(kuangnum)
print(kuangaccuracy)
cPickle.dump(model,open("./model8.18.pkl","wb"))

@tangjie77wd You need to add K.learning_phase() to the list of inputs when creating your K.function as explained in my comment.
So something like
K.function([K.learning_phase(), origin_model.layers[0].input],origin_model.layers[13].output)

@julienr Thanks for your comment.I have run it again after changed like this:get_feature = K.function([K.learning_phase(), origin_model.layers[0].input],origin_model.layers[13].output),but I got the same error.
`
File "E:/Keras_Code/luobin_cnn-svm8.18.py", line 59, in
get_feature = K.function([K.learning_phase(), origin_model.layers[0].input],origin_model.layers[13].output)
File "C:\Anaconda2\lib\site-packages\keras\backend\theano_backend.py", line 572, in function
return Function(inputs, outputs, updates=updates, **kwargs)
File "C:\Anaconda2\lib\site-packages\keras\backend\theano_backend.py", line 558, in init
**kwargs)
File "C:\Anaconda2\lib\site-packages\theano\compile\function.py", line 320, in function
output_keys=output_keys)
File "C:\Anaconda2\lib\site-packages\theano\compile\pfunc.py", line 479, in pfunc
output_keys=output_keys)
File "C:\Anaconda2\lib\site-packages\theano\compile\function_module.py", line 1776, in orig_function
output_keys=output_keys).create(
File "C:\Anaconda2\lib\site-packages\theano\compile\function_module.py", line 1428, in init
accept_inplace)
File "C:\Anaconda2\lib\site-packages\theano\compile\function_module.py", line 177, in std_fgraph
update_mapping=update_mapping)
File "C:\Anaconda2\lib\site-packages\theano\gof\fg.py", line 171, in init
self.import_r(output, reason="init")
File "C:\Anaconda2\lib\site-packages\theano\gof\fg.py", line 360, in import_r
self.import(variable.owner, reason=reason)
File "C:\Anaconda2\lib\site-packages\theano\gof\fg.py", line 474, in import
r)
MissingInputError: ("An input of the graph, used to compute DimShuffle{x,x,x,x}(keras_learning_phase), was not provided and not given a value.Use the Theano flag exception_verbosity='high',for more information on this error.", keras_learning_phase)`

@tangjie77wd It might have something to do with the use of cPickle. Maybe try running compile on your model after you have loaded it. Or try with model.save or model.to_json.
The following works for me (keras master) - this is a minimal example based on your code
import os
os.environ['KERAS_BACKEND'] = 'theano'

from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation, Flatten
from keras.layers.convolutional import Convolution2D, MaxPooling2D
from keras.optimizers import SGD

model = Sequential()
model.add(Convolution2D(32, 5, 5, border_mode='valid', input_shape=(1,60,60)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.3))

model.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors
model.add(Dense(128))
model.add(Activation('relu'))
model.add(Dropout(0.3))
model.add(Dense(1))
model.add(Activation('sigmoid'))

sgd = SGD(lr=0.005, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='binary_crossentropy', optimizer=sgd, metrics=["accuracy"])

import keras.backend as K
f = K.function([K.learning_phase(), model.layers[0].input], [model.layers[4].output])

Maybe you want to try AETROS which visualises weights and convolutional layer out of the box using a simple callback. https://youtu.be/_HsuYywGmXA?t=127,

However,as you know,i wanna get the feature of data after Flatten layer finally,so when i run like below:
f = K.function([K.learning_phase(), origin_model.layers[0].input], [origin_model.layers[13].output])
feature=f(data)
It shows error as the following shows:
File "E:/Keras_Code/luobin_cnn-svm8.18.py", line 79, in
feature=f(data)
File "C:\Anaconda2\lib\site-packages\keras\backend\theano_backend.py", line 561, in call
assert type(inputs) in {list, tuple}
Aha,'feature=f(data)' cause the error again!I am wandering if the shape of data is conflict with the origin_model.layer[0].input.

However,as you know,i wanna get the feature of data after Flatten layer finally,so when i run like below:
f = K.function([K.learning_phase(), origin_model.layers[0].input], [origin_model.layers[13].output])
feature=f(data)
It shows error as the following shows:
File "E:/Keras_Code/luobin_cnn-svm8.18.py", line 79, in
feature=f(data)
File "C:\Anaconda2\lib\site-packages\keras\backend\theano_backend.py", line 561, in call
assert type(inputs) in {list, tuple}
Aha,'feature=f(data)' causes the error again!I am wandering if the shape of data is conflict with the origin_model.layer[0].input.

Oh,I changed 'feature=f(data)' into 'feature=f([data])'.Now the error is:
TypeError: ('Bad input argument to theano function with name "C:\Anaconda2\lib\site-packages\keras\backend\theano_backend.py:558"  at index 0(0-based)', 'Wrong number of dimensions: expected 0, got 4 with shape (1040L, 1L, 60L, 60L).')

hi @kallewoof  ,I use the graph model.
like this,

model = graph()
model.add_input(name='input0',input_shape=())
model.add_node(Convolution2D(),name='c1',input='input0')
.......

And i want to see the output of the c1,Then

getFeatureMap = theano.function(model.inputs['input0'].input,model.nodes['c1'].get_output(train=False),
allow_input_downcast=True)
But it show me that
TypeError: list indices must be integers, not str
Do you give me some advices? Thanks.

Unfortunately I haven't used Keras for some time, but hopefully someone else can help out!

hi @julienr ,i use the graph model ,refer to #41
like this,

model = graph()
model.add_input(name='input0',input_shape=())
model.add_node(Convolution2D(),name='c1',input='input0')
.......

And i want to see the output of the c1,Then

getFeatureMap = theano.function(model.inputs['input0'].input,model.nodes['c1'].get_output(train=False),
allow_input_downcast=True)
But it show me that
TypeError: list indices must be integers, not str
Do you give me some advices? Thanks.

hi @saj1919  ,i use the graph model ,refer to #41
like this,

model = graph()
model.add_input(name='input0',input_shape=())
model.add_node(Convolution2D(),name='c1',input='input0')
.......

And i want to see the output of the c1,Then

getFeatureMap = theano.function(model.inputs['input0'].input,model.nodes['c1'].get_output(train=False),
allow_input_downcast=True)
But it show me that
TypeError: list indices must be integers, not str
Do you give me some advices? Thanks.

hi @guy4261  ,i use the graph model ,refer to #41  and my keras is 1.0.3
like this,

model = graph()
model.add_input(name='input0',input_shape=())
model.add_node(Convolution2D(),name='c1',input='input0')
.......

And i want to see the output of the c1,Then

getFeatureMap = theano.function(model.inputs['input0'].input,model.nodes['c1'].get_output(train=False),
allow_input_downcast=True)
But it show me that
TypeError: list indices must be integers, not str
Do you give me some advices? Thanks.

@julienr
I tried your code:
inputs = [theano.learning_phase()] + model.inputs
convout1_f = theano.function(inputs, convout1.get_output(train=False))

but I got the error:

AttributeError: 'module' object has no attribute 'learning_phase'

Do you perhaps know why I got this error? Thank you.

@matchifang This is because learning_phase is in keras.backend, not in theano.
import keras.backend as K
inputs = [K.learning_phase()] + model.inputs
convout1_f = K.function(inputs, convout1.get_output(train=False))

Hi @julienr , I'm trying to implement your code (https://github.com/julienr/ipynb_playground/blob/master/keras/convmnist/keras_cnn_mnist_v1.ipynb) and apply it to obtain the featrures maps of a CNN. I'm loading a model I've previously trained. I get the following error with this code:
import os
os.environ['THEANO_FLAGS']='mode=FAST_RUN,device=gpu,floatX=float32'
# This gives a 'perform' error in compile
#os.environ['THEANO_FLAGS']='mode=FAST_COMPILE,device=gpu1,floatX=float32'
#os.environ['THEANO_FLAGS']='device=gpu0'
import pylab as pl
import matplotlib.cm as cm
import numpy as np
from keras.layers import Activation
from keras.models import load_model
from keras import backend as K

model = load_model('model.149-0.18.hdf5')
X_test = np.load('test_X_1inP.npy')
convout1 = Activation('relu')
inputs = [K.learning_phase()] + model.inputs

_convout1_f = K.function(inputs, [convout1.output])
def convout1_f(X):
    # The [0] is to disable the training phase flag
    return _convout1_f([0] + [X])

Traceback (most recent call last):

  File "<ipython-input-25-8723074daa75>", line 3, in <module>
    _convout1_f = K.function(inputs, [convout1.output])

  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 776, in output
    'hence the notion of "layer output" '

Exception: Layer activation_2 has multiple inbound nodes, hence the notion of "layer output" is ill-defined. Use `get_output_at(node_index)` instead.

Is your code working for any loaded model?
Thank you very much.

@Mako18 I'm note sure what's the problem. Can you try using convout1.get_output_at(0) instead of convout1.output as the error message suggets ?

Hi, @julienr I have tried also that, but it gives me another error:
Traceback (most recent call last):

  File "<ipython-input-33-4c9014fadc67>", line 1, in <module>
    _convout1_f = K.function(inputs, [convout1.get_output_at(0)])

  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 674, in get_output_at
    'output')

  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 636, in _get_node_attribute_at_index
    'and thus has no defined ' + attr_name + '.')

Exception: The layer has never been called and thus has no defined output.

Then I tried to apply the code to a model I trained in that moment and the error was other. The issue is that I am interested in apply it to a loaded model, previously trained. I guess I am missing something, sorry if it is too dumb, but I am relatively new to Keras and Python. Thanks.

@Mako18 I'm not sure where this error comes from. Does calling model.compile solve the problem ?

@julienr It doesn't, I tried compiling the model before the line _convout1_f = K.function(inputs, [convout1.get_output_at(0)]) and I'm not sure about how can I fix it.

How to plot convolution 1d weights with 64 filters

@Mako18 you are not using the Activation layer from your loaded model but creating a new one with convout1 = Activation('relu').
Instead try this code:
model = load_model('model.149-0.18.hdf5')
layer_idx = [idx for idx, layer in enumerate(model.layers) if layer.name == 'convolution2d_1'][0]
convout1 = model.layers[layer_idx]

You may have to replace convolution2d_1 with the name of your first conv layer.

Hi @xge, in that case I get another error, although it make sense your code. My inputs are three separated images, each one with 2 channels, so the input of my network is [XL, XR, XF], where each structure has dimensions (2100, 2, 10, 8). The error I get in this case is:
Traceback (most recent call last):

  File "<ipython-input-43-9b29e8db04b3>", line 1, in <module>
    W = model.layers[0].W.get_value(borrow=True)

AttributeError: 'Merge' object has no attribute 'W'

After the code
# Visualize weights
W = model.layers[0].W.get_value(borrow=True)
W = np.squeeze(W)
print("W shape : ", W.shape)

pl.figure(figsize=(15, 15))
pl.title('conv1 weights')
nice_imshow(pl.gca(), make_mosaic(W, 6, 6), cmap=cm.binary)

Also in order to get the code working I had to set X=X_test[i:i+1, 1, :, :] when trying to visualize the first layer of convolutions on an input image, but couldn't introduce the two channels of my input.

@tangjie77wd  @julienr did you solve your error problem. i am using same model and got same error during feature extraction.
can you tell how can solve this problem

@tangjie77wd you can try this
def get_activations(model, layer, X):
get_activations = K.function([model.layers[0].input, K.learning_phase()], [model.layers[9].output,])
activations = get_activations([X,0])
return activations

@myk1603 great, thanks!

Visualizing the average activation values of the input features and Saliency map for a chosen sample
my network is given below

define the network

model = Sequential()
model.add(LSTM(4,input_dim=42)) # try using a GRU instead, for fun
model.add(Dropout(0.1))
model.add(Dense(5))
model.add(Activation('softmax'))
#Taylor expansion and compute first partial derivative of the classification results
from keras import backend as K
import theano
def compile_saliency_function(model):
"""
Compiles a function to compute the saliency maps and predicted classes
for a given minibatch of input images.
"""
inp = model.layers[0].input
print("-----------------------input-----------------------------")
print(inp)
outp = model.layers[-1].output
print("-----------------------output----------------------------")
print(outp)
max_outp = K.T.max(outp, axis=1)
print(max_outp)
saliency = K.gradients(K.sum(max_outp), inp)
print(saliency)
max_class = K.T.argmax(outp, axis=1)
print(max_class)
v1 = K.function([inp, K.learning_phase()], [saliency, max_class])
return v1
print(([X_train[:20], 0])[0])
v = compile_saliency_function(model)([X_train[:5], 0])[0]
print(v)
Sailnce map gives
[[[ -8.015e-29 6.428e-28 -1.365e-29 3.198e-30 -1.229e-29 -2.009e-30
-1.273e-30 -2.512e-29 -6.153e-28 3.688e-29 2.205e-28 4.094e-28
5.667e-28 -1.401e-27 1.892e-28 5.810e-30 1.736e-29 7.379e-29
-2.122e-28 8.063e-29 3.660e-31 1.458e-31 9.196e-28 -1.335e-30
1.608e-30 -8.264e-29 -1.094e-28 1.099e-29 6.411e-30 1.134e-28
3.281e-29 3.869e-29 -1.276e-30 -9.509e-31 8.313e-29 4.073e-29
-8.345e-30 3.455e-29 -1.100e-28 -8.354e-29 -5.175e-29 1.583e-29]]
[[ -8.015e-29 6.428e-28 -1.365e-29 3.198e-30 -1.229e-29 -2.009e-30
-1.273e-30 -2.512e-29 -6.153e-28 3.688e-29 2.205e-28 4.094e-28
5.667e-28 -1.401e-27 1.892e-28 5.810e-30 1.736e-29 7.379e-29
-2.122e-28 8.063e-29 3.660e-31 1.458e-31 9.196e-28 -1.335e-30
1.608e-30 -8.264e-29 -1.094e-28 1.099e-29 6.411e-30 1.134e-28
3.281e-29 3.869e-29 -1.276e-30 -9.509e-31 8.313e-29 4.073e-29
-8.345e-30 3.455e-29 -1.100e-28 -8.354e-29 -5.175e-29 1.583e-29]]
[[ -8.015e-29 6.428e-28 -1.365e-29 3.198e-30 -1.229e-29 -2.009e-30
-1.273e-30 -2.512e-29 -6.153e-28 3.688e-29 2.205e-28 4.094e-28
5.667e-28 -1.401e-27 1.892e-28 5.810e-30 1.736e-29 7.379e-29
-2.122e-28 8.063e-29 3.660e-31 1.458e-31 9.196e-28 -1.335e-30
1.608e-30 -8.264e-29 -1.094e-28 1.099e-29 6.411e-30 1.134e-28
3.281e-29 3.869e-29 -1.276e-30 -9.509e-31 8.313e-29 4.073e-29
-8.345e-30 3.455e-29 -1.100e-28 -8.354e-29 -5.175e-29 1.583e-29]]
[[ -1.190e-28 7.039e-28 -3.475e-30 3.815e-30 -1.410e-29 -2.210e-30
-1.390e-30 -3.224e-29 -7.007e-28 3.122e-29 2.429e-28 4.564e-28
6.228e-28 -1.537e-27 1.703e-28 1.865e-29 1.983e-29 7.343e-29
-2.412e-28 7.795e-29 3.947e-31 1.972e-31 9.979e-28 -1.683e-30
1.429e-30 -1.043e-28 -1.336e-28 1.509e-29 6.298e-30 1.371e-28
4.295e-29 7.593e-29 -1.360e-30 -5.704e-31 9.312e-29 8.257e-29
-3.896e-30 4.088e-29 -1.375e-28 -1.039e-28 -6.191e-29 1.965e-29]]
[[ -1.188e-28 7.032e-28 -3.503e-30 3.810e-30 -1.408e-29 -2.208e-30
-1.388e-30 -3.220e-29 -6.999e-28 3.121e-29 2.427e-28 4.559e-28
6.221e-28 -1.535e-27 1.702e-28 1.860e-29 1.980e-29 7.337e-29
-2.409e-28 7.790e-29 3.943e-31 1.969e-31 9.969e-28 -1.681e-30
1.429e-30 -1.042e-28 -1.334e-28 1.507e-29 6.294e-30 1.369e-28
4.289e-29 7.576e-29 -1.359e-30 -5.711e-31 9.302e-29 8.237e-29
-3.906e-30 4.083e-29 -1.373e-28 -1.038e-28 -6.183e-29 1.962e-29]]]
#Visualizing the average activation values of the input features for 5 samples of class 0
def get_activations(model, layer, X_batch):
get_activations = K.function([model.layers[0].input, K.learning_phase()], model.layers[layer].output)
activations = get_activations([X_batch,0])
return activations
my_featuremaps = get_activations(model, 0, ([X_train[:5], 0])[0])
print(my_featuremaps)
np.savetxt('featuremap', my_featuremaps)
activation values
[[ 0.762 -0.122 0.044 -0.758]
[ 0.762 -0.122 0.044 -0.758]
[ 0.762 -0.122 0.044 -0.758]
[ 0.762 -0.038 0.043 -0.752]
[ 0.762 -0.038 0.043 -0.752]
[ 0.762 -0.038 0.043 -0.752]
[ 0.125 -0. 0.038 -0.083]
[ 0.762 -0.121 0.044 -0.758]
[ 0.762 -0.038 0.043 -0.752]
[ 0.762 -0.433 0.091 -0.755]]
Could you please tell that the followed methid is correct if So could you please tell how to generate the plots fig 3 fig 6 of the paper titled "Empowering Convolutional Networks for Malware
Classification and Analysis"

Hello @julienr @johnny5550822 @tangjie77wd
data input for CNN
X_train = (2012L, 3L, 200L, 200L)
Y_train = (2012L, 2L) #label after encoding for CNN
#Store features after CNN training
get_activations = K.function([model.layers[0].input, K.learning_phase()], [model.layers[9].output,])
feature = get_activations([X_train,0])
#Now I have feature vector
#features = float (2012L, 128L)
y_train = (2012L,)
#I want to use this feature vector with SVM anyone can help me?
#currently, I am trying this code
svcClf = SVC(C=1.0,kernel="rbf",cache_size=975)
scaler = MinMaxScaler() #transform features by scaling each features to a given range
feature = scaler.fit_transform(feature)
svcClf.fit(feature, y_train)
#But I got error
with scaler
ValueError: Found array with dim 3. MinMaxScaler expected <= 2.
Without scaler
ValueError: Found array with dim 3. Estimator expected <= 2

Hi, @julienr I have solved the errors I had, so now I am able to visualize the weights of my last convolutional layer, but not the convolutions. It gives me this error:
  File "C:\Users\Macarena\Anaconda2\lib\site-packages\theano\compile\function_module.py", line 845, in __call__
    self.inv_finder[c]))

TypeError: Missing required input: convolution2d_input_20´´´

My model start with three different networks each one with a convolutional layer (starting with `convolution2d_21´). After that convolution I apply a Merge layer and after that a convolutional layer (the last one, `convolution2d_24). One of my problems is that I am not able to set the first convolutional layer (of one of the three starting branches) as `convout1´, I would like to solve that. Also, my model takes as inputs three different images of two channels with 10x8 dimension, so the input structure is `(nb_samples, 3L, 2L, 10L, 8L)´, how can I modify the `convout1_f´ definition to work with this case?

@Mako18 It's hard to tell without code to test. I think convout1_f should be a function taking 3 inputs (which are the 3 inputs to your top-level networks).
Can you provide a minimal example of the network construction and visualization that reproduces the problem ?

More generally you can visualise the output/activations of every layer of your model. I wrote an example with MNIST to show how here:
https://github.com/philipperemy/keras-visualize-activations
So far it's the less painful I've seen.

@philipperemy Thank you for the awesome get_activations function. I would like to post it here for people who just want a copy paste implementation. It would be great to have this as a functionality in Keras ! If someone from the Keras is willing to review, I am ready to work on this, write the necessary documentation, tests etc to add this.
def get_activations(model, model_inputs, print_shape_only=True, layer_name=None):
    import keras.backend as K
    print('----- activations -----')
    activations = []
    inp = model.input

    model_multi_inputs_cond = True
    if not isinstance(inp, list):
        # only one input! let's wrap it in a list.
        inp = [inp]
        model_multi_inputs_cond = False

    outputs = [layer.output for layer in model.layers if
               layer.name == layer_name or layer_name is None]  # all layer outputs

    funcs = [K.function(inp + [K.learning_phase()], [out]) for out in outputs]  # evaluation functions

    if model_multi_inputs_cond:
        list_inputs = []
        list_inputs.extend(model_inputs)
        list_inputs.append(1.)
    else:
        list_inputs = [model_inputs, 1.]

    # Learning phase. 1 = Test mode (no dropout or batch normalization)
    # layer_outputs = [func([model_inputs, 1.])[0] for func in funcs]
    layer_outputs = [func(list_inputs)[0] for func in funcs]
    for layer_activations in layer_outputs:
        activations.append(layer_activations)
        if print_shape_only:
            print(layer_activations.shape)
        else:
            print(layer_activations)
    return activations

# activations = get_activations(model, X_test)
# print(activations[0])

@sahmed95 @philipperemy
Thanks a lot for sharing!
One simple issue,
   # Learning phase. 1 = Test mode (no dropout or batch normalization)
I found in Keras (2.0.2)
    The learning phase flag is a bool tensor (0 = test, 1 = train)
And in practice I set the flag to 0 will get same output for same samples.

@zhhongzhi good catch! Maybe I did a mistake there.
I didn't test it for dropout or batch normalization so the result was always the same.
Please open a PR at:
https://github.com/philipperemy/keras-visualize-activations
Thanks a lot!

@philipperemy Thank you for the awesome get_activations function. I would like to post it here for people who just want a copy paste implementation. It would be great to have this as a functionality in Keras ! If someone from the Keras is willing to review, I am ready to work on this, write the necessary documentation, tests etc to add this.
def get_activations(model, model_inputs, print_shape_only=True, layer_name=None):
    import keras.backend as K
    print('----- activations -----')
    activations = []
    inp = model.input

    model_multi_inputs_cond = True
    if not isinstance(inp, list):
        # only one input! let's wrap it in a list.
        inp = [inp]
        model_multi_inputs_cond = False

    outputs = [layer.output for layer in model.layers if
               layer.name == layer_name or layer_name is None]  # all layer outputs

    funcs = [K.function(inp + [K.learning_phase()], [out]) for out in outputs]  # evaluation functions

    if model_multi_inputs_cond:
        list_inputs = []
        list_inputs.extend(model_inputs)
        list_inputs.append(1.)
    else:
        list_inputs = [model_inputs, 1.]

    # Learning phase. 1 = Test mode (no dropout or batch normalization)
    # layer_outputs = [func([model_inputs, 1.])[0] for func in funcs]
    layer_outputs = [func(list_inputs)[0] for func in funcs]
    for layer_activations in layer_outputs:
        activations.append(layer_activations)
        if print_shape_only:
            print(layer_activations.shape)
        else:
            print(layer_activations)
    return activations

# activations = get_activations(model, X_test)
# print(activations[0])

Hi quantshah, can the function work in TF2.0? When I run the function, I got the follow error

ValueError                                Traceback (most recent call last)
Cell In [21], line 1
----> 1 activations = utils.get_activations(model, x_test[test_image_idx])
File ~/codes/hands-on-transfer-learning-with-python/notebooks/Ch06 - Image Recognition and Classification/cnn_utils.py:88, in get_activations(model, model_inputs, print_shape_only, layer_name)
84 outputs = [layer.output for layer in model.layers if
85            layer.name == layer_name or layer_name is None]
87 # evaluation functions
---> 88 funcs = [K.function(inp + [K.learning_phase()], [out]) for out in outputs]
90 if model_multi_inputs_cond:
91     list_inputs = []
File ~/codes/hands-on-transfer-learning-with-python/notebooks/Ch06 - Image Recognition and Classification/cnn_utils.py:88, in (.0)
84 outputs = [layer.output for layer in model.layers if
85            layer.name == layer_name or layer_name is None]
87 # evaluation functions
---> 88 funcs = [K.function(inp + [K.learning_phase()], [out]) for out in outputs]
90 if model_multi_inputs_cond:
91     list_inputs = []
File ~/anaconda3/envs/tf2/lib/python3.8/site-packages/tensorflow/python/keras/backend.py:4069, in function(inputs, outputs, updates, name, **kwargs)
4067 from tensorflow.python.keras import models  # pylint: disable=g-import-not-at-top
4068 from tensorflow.python.keras.utils import tf_utils  # pylint: disable=g-import-not-at-top
-> 4069 model = models.Model(inputs=inputs, outputs=outputs)
4071 wrap_outputs = isinstance(outputs, list) and len(outputs) == 1
4072 def func(model_inputs):
File ~/anaconda3/envs/tf2/lib/python3.8/site-packages/tensorflow/python/training/tracking/base.py:517, in no_automatic_dependency_tracking.._method_wrapper(self, *args, **kwargs)
515 self._self_setattr_tracking = False  # pylint: disable=protected-access
516 try:
--> 517   result = method(self, *args, **kwargs)
518 finally:
519   self._self_setattr_tracking = previous_value  # pylint: disable=protected-access
File ~/anaconda3/envs/tf2/lib/python3.8/site-packages/tensorflow/python/keras/engine/functional.py:120, in Functional.init(self, inputs, outputs, name, trainable, **kwargs)
118 generic_utils.validate_kwargs(kwargs, {})
119 super(Functional, self).init(name=name, trainable=trainable)
--> 120 self._init_graph_network(inputs, outputs)
File ~/anaconda3/envs/tf2/lib/python3.8/site-packages/tensorflow/python/training/tracking/base.py:517, in no_automatic_dependency_tracking.._method_wrapper(self, *args, **kwargs)
515 self._self_setattr_tracking = False  # pylint: disable=protected-access
516 try:
--> 517   result = method(self, *args, **kwargs)
518 finally:
519   self._self_setattr_tracking = previous_value  # pylint: disable=protected-access
File ~/anaconda3/envs/tf2/lib/python3.8/site-packages/tensorflow/python/keras/engine/functional.py:157, in Functional._init_graph_network(self, inputs, outputs)
154   if any(not hasattr(tensor, '_keras_history') for tensor in self.outputs):
155     base_layer_utils.create_keras_history(self._nested_outputs)
--> 157 self._validate_graph_inputs_and_outputs()
159 # A Network does not create weights of its own, thus it is already
160 # built.
161 self.built = True
File ~/anaconda3/envs/tf2/lib/python3.8/site-packages/tensorflow/python/keras/engine/functional.py:688, in Functional._validate_graph_inputs_and_outputs(self)
686 if not hasattr(x, '_keras_history'):
687   cls_name = self.class.name
--> 688   raise ValueError('Input tensors to a ' + cls_name + ' ' +
689                    'must come from tf.keras.Input. '
690                    'Received: ' + str(x) +
691                    ' (missing previous layer metadata).')
692 # Check that x is an input tensor.
693 # pylint: disable=protected-access
694 layer = x._keras_history.layer
ValueError: Input tensors to a Functional must come from tf.keras.Input. Received: 0 (missing previous layer metadata).


text:
Keras(Python) stops working after first epoch with any error

Hello,
I have latest version of theano and keras and I am trying to run script from keras from this page for learning purposes:
https://machinelearningmastery.com/develop-word-based-neural-language-models-python-keras/
This first example of full code from the page.
When I run the code it work ok untill it starts training the model. Then python just stops executing without reporting any error. What could be the problem?
########################################################
Using Theano backend.
WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
Vocabulary Size: 22
Total Sequences: 24

Layer (type)                 Output Shape              Param #
embedding_1 (Embedding)      (None, 1, 10)             220

lstm_1 (LSTM)                (None, 50)                12200

dense_1 (Dense)              (None, 22)                1122
Total params: 13,542
Trainable params: 13,542
Non-trainable params: 0

None
Epoch 1/500
###############################################
Thanks

Sounds like a segfault (or similar) within Theano itself.

Thanks for the answer. Is there any way to debug and determine what the problem is? Because I do not get any error - such as segfault error. It just stops working.
Is there any way to turn on log or something.
Also, does anyone have any working short example of using LSTM example in Keras, so I can check if the example works for me.

You will find plenty of LSTM examples in the Keras examples folder.
Since the issue is with Theano, I recommend you try using the TensorFlow backend instead.

I prefer Theano because I run python 2.7 and for Tensorflow I need 3.4+ (I think). So it would be the best if I managed for the Theano to work somehow.
Thanks I have found the example and will try some of them.
I tried another example not in the keras folder and I got this error:
WARNING (theano.gof.compilelock): Overriding existing lock by dead process '2092' (I am process '7072')

I tried a few examples and I have found that this example behaves the same way: - https://github.com/keras-team/keras/blob/master/examples/imdb_lstm.py
It stops working after the first epoch as well:

It sounds like you have a bad installation of Theano. Uninstall everything,
reinstall.
Or alternatively, use TensorFlow (which works with Python 2.7, too).
On 26 February 2018 at 17:57, Oxi84 notifications@github.com wrote:

I tried a few examples and I have found that this example behaves the same
way: - https://github.com/keras-team/keras/blob/master/examples/
imdb_lstm.py
It stops working after the first epoch as well:
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/keras-team/keras/issues/9497#issuecomment-368719363,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AArWbxAd7BUyf6OoThSN-RFRBJq5Spuqks5tY2D5gaJpZM4SUKEM
.

Will try to uninstall it. Ordinary ANNs work fine with such Theano - just LSTM do not work for soem reason.
I use windows as well so I guess it is not possible to install it without complications: - https://stackoverflow.com/questions/45316569/how-to-install-tensorflow-on-python-2-7-on-windows

I did:
pip uninstall theano
pip uninstall keras
pip install keras
pip install Theano
Still I get the same error. On another PC of mine it works well. But this one is much slower.

Also I get Windows error in event viewer - seems like faulty module is Theano:

Woks now but slow - i got this warning: WARNING (theano.configdefaults): g++ not detected ! Theano will be unable to execute optimized C-implementations (for both CPU and GPU) and will default to Python implementations. Performance will be severely degraded. To remove this warning, set Theano flags cxx to an empty string.
WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
Is there any way to maanually set gc++ compiler. I installed MinGW, but theano still doesn't detect it. I would like to put the patch to the .exe somewhere.
Solution was:I reinstalled c+ compiler, it was cygwin (TMD or something ) and I have deinstalled it.  This was error cause.
Now works, but it is really slow.

Because you should install g++. Theano only works (fast) with g++. And I
don't recommend using Theano with Windows.
On 26 February 2018 at 20:58, Oxi84 notifications@github.com wrote:

Woks now but slow - i got this warning: WARNING (theano.configdefaults):
g++ not detected ! Theano will be unable to execute optimized
C-implementations (for both CPU and GPU) and will default to Python
implementations. Performance will be severely degraded. To remove this
warning, set Theano flags cxx to an empty string.
WARNING (theano.tensor.blas): Using NumPy C-API based implementation for
BLAS functions.
Is there any way to maanually set gc++ compiler. I installed MinGW, but
theano still doesn't detect it. I would like to put the patch to the .exe
somewhere.
Solution was:I reinstalled c+ compiler, it was cygwin (TMD or something )
and I have deinstalled it. This was error cause.
Now works, but it is really slow.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/keras-team/keras/issues/9497#issuecomment-368747791,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AArWbzeTh4p0y0DE-1H_d1XY0DrR-XNPks5tY4uKgaJpZM4SUKEM
.

I know, i installed  I installed MinGW but do not know where to add it to patch.

source: gharchive/issue

repo: keras-team/keras

license_type: permissive

license_source: bigquery


text:
Add Get All Initializers in the Form of String

Add a new API 'initializers.get_all_initializers", which returns all initializers in the form of string.
As mentioned in #15393 , it can facilitate the process of fine-tuning.

Thanks for the PR. I would recommend that you keep this functionality as a utility function outside of Keras. You could just iterate over dir(tf.keras.initializers) and retrieve x.__name__ for functions, or x.__class__.__name__ for classes.

Thanks for the PR. I would recommend that you keep this functionality as a utility function outside of Keras. You could just iterate over dir(tf.keras.initializers) and retrieve x.__name__ for functions, or x.__class__.__name__ for classes.

Actually, it is pretty ugly to retrieve functions by x.__name__ or classes by x.__class__.__name__. And few Keras users have the same deep understanding of Python as you do.

source: gharchive/pull-request

authors: WingsBrokenAngel fchollet


text:
Accurarcy() does not work, but 'accuracy' does

System information.

Have I written custom code (as opposed to using a stock example script provided in Keras): not really
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 22.04
TensorFlow installed from (source or binary): pip install tensorflow
TensorFlow version (use command below): 2.13.0
Python version: 3.11.5
Bazel version (if compiling from source): -
GPU model and memory: does not matter
Exact command to reproduce: run script below

Describe the problem.
If I write Accurarcy() in the metrics list, it does not work. But the String accuracy does work. According to the docs, bith sould work. See example code below.
Describe the current behavior.
469/469 [==============================] - 3s 6ms/step - loss: 0.2548 - accuracy: 0.0000e+00

Describe the expected behavior.
469/469 [==============================] - 3s 6ms/step - loss: 0.2540 - accuracy: 0.9260

Contributing.

Do you want to contribute a PR? (yes/no): no
If yes, please read this page for instructions
Briefly describe your candidate solution(if contributing): -

Standalone code to reproduce the issue.
import numpy as np
from keras import Sequential
from keras.datasets import mnist
from keras.src import activations
from keras.src.layers import Dense
from keras.src.losses import CategoricalCrossentropy
from keras.src.metrics import Accuracy
from keras.src.optimizers import RMSprop
from keras.src.utils import to_categorical

def preprocess_images(images):
    s = images.shape
    return images.reshape((s[0], s[1] * s[2])).astype(dtype=np.float32) / 255

if __name__ == '__main__':
    (train_images, train_labels), (test_images, test_labels) = mnist.load_data()
    processed_train_images = preprocess_images(train_images)
    processed_test_images = preprocess_images(test_images)
    processed_train_labels = to_categorical(y=train_labels)
    processed_test_labels = to_categorical(y=test_labels)

    network = Sequential()
    network.add(layer=Dense(units=512, activation=activations.relu, input_shape=(28 * 28, )))
    network.add(layer=Dense(units=10, activation=activations.softmax))

    network.compile(optimizer=RMSprop(), loss=CategoricalCrossentropy(), metrics=[Accuracy()])  # this does not work
    # network.compile(optimizer=RMSprop(), loss=CategoricalCrossentropy(), metrics=['accuracy'])  # this works

    network.fit(x=processed_train_images, y=processed_train_labels, epochs=1, batch_size=128)

Source code / logs.
Nothing.

@LostInDarkMath,
Apologies for the delay. Here in the code you are trying to use loss=CategoricalCrossentropy(), where Accuracy is not the better option. Instead you can use keras.metrics.CategoricalAccuracy for the accurate result. I tried to execute with keras.metrics.CategoricalAccuracy and it provided the required output. Kindly find the gist of it here.

https://www.tensorflow.org/api_docs/python/tf/keras/metrics/CategoricalAccuracy
Thank you!

Thank your for the clarification.
That means that the string 'accuracy' is resolved differently depending on the loss function? Or does 'accuracy' always resolve to eras.metrics.CategoricalAccuracy? And where is dthis behavior documented? Is there something like a mapping table that visualizes this mapping? Would be very nice :)

@LostInDarkMath
Here the following API are technically same.
keras.metrics.Accuracy
keras.metrics.BinaryAccuracy

From doc,

keras.metrics.Accuracy(name="accuracy", ...)
keras.metrics.BinaryAccuracy(name="binary_accuracy", ...)
Calculates how often predictions equal labels.
This metric creates two local variables, total and count that are used to compute the frequency with which y_pred matches y_true. This frequency is ultimately returned as binary accuracy: an idempotent operation that simply divides total by count.

And when we use string identifier, for example [accuracy], later it converts to the appropriate metrics based on the labels and logits, source.

When you pass the
strings 'accuracy' or 'acc', we convert this to one of
tf.keras.metrics.BinaryAccuracy,
tf.keras.metrics.CategoricalAccuracy,
tf.keras.metrics.SparseCategoricalAccuracy based on the shapes
of the targets and of the model output. We do a similar

@tilakrayal
Apart from the abvove clarification, there may be potential bug. Please check this question on stack overflow. That is very similar to this issue.

From the model.copile document:

List of metrics to be evaluated by the model during training and testing. Each of this can be a string (name of a built-in function), function or a tf.keras.metrics.Metric instance. See tf.keras.metrics. Typically you will use metrics=['accuracy']. A function is any callable with the signature result = fn(y_true,y_pred). To specify different metrics for different outputs of a multi-output model, you could also pass a dictionary, such as metrics={'output_a':'accuracy', 'output_b':['accuracy', 'mse']}. You can also pass a list to specify a metric or a list of metrics for each output, such as metrics=[['accuracy'], ['accuracy', 'mse']] or metrics=['accuracy', ['accuracy', 'mse']]. When you pass the strings 'accuracy' or 'acc', we convert this to one of tf.keras.metrics.BinaryAccuracy, tf.keras.metrics.CategoricalAccuracy, tf.keras.metrics.SparseCategoricalAccuracy based on the shapes of the targets and of the model output. We do a similar conversion for the strings 'crossentropy' and 'ce' as well. The metrics passed here are evaluated without sample weighting; if you would like sample weighting to apply, you can specify your metrics via the weighted_metrics argument instead.

When you use metrics.Accuracy, it resolves to BinaryAccuracy as stated in the doc of Accuracy.  Whereas, when using string accuracy it resolves to BinaryAccuracy or CategoricalAccuracy, based on the target shape.  This was also explained by @innat above. This may be confusing, so we recommend either explicitly using BinaryAccuracy or CategoricalAccuracy if you don't want to use the string accuracy for the metrics.

The same behavior exists in Keras 3 as well.  Should we deprecate use of metrics.Accuracy or mimic its usage to string metric of accuracy?

The same behavior exists in Keras 3 as well.  Should we deprecate use of metrics.Accuracy or mimic its usage to string metric of accuracy?

The metrics.Accuracy can be removed. It makes confusion, especially among the beginners a lot.

authors: LostInDarkMath innat sachinprasadhs sampathweb tilakrayal

repo: keras-team/tf-keras

license_source: github-api


text:
Create holdout set

This PR will select some paper to have an holdout set.
At the moment, as the data set is small, we will use all the documents for create the final models, however we will keep a fixed holdout set to have a more strict and precise evaluation.
The holdout set was created using an automatic script and re-balanced based on the distribution of entities between training and holdout set.
The python script to reproduce the holdout dataset are contained under scripts.
The statistics about the training/holdout set can be found in:

https://github.com/kermitt2/grobid-quantities/tree/feature/holdout-set/resources/dataset/quantities
https://github.com/kermitt2/grobid-quantities/tree/feature/holdout-set/resources/dataset/units
https://github.com/kermitt2/grobid-quantities/tree/feature/holdout-set/resources/dataset/values

Coverage remained the same at 27.67% when pulling f682de6ac0bc2470d6a27634f58f0a0400ebea13 on feature/holdout-set into 0957bc631017ca3c603bb394d53af4b9643720d3 on master.

I think this is ready to merge

authors: coveralls lfoppiano


text:
CLI: return a nonzero status when validation fails

This tool is great, but so far int merely prints errors and warnings, which makes it difficult to evaluate the validation status e.g. by a CI script or pre-commit hook.
A naïve implementation might just evaluate the list of errors and warnings and exit the process with a non-zero status:
    const errorsAndWarnings = validateSchema(targetObj, options)
    if (errorsAndWarnings?.length > 0) {
      process.exit(1)
    }

@lwille Looks like a useful improvement. But would like to have it added as an optional param that can be passed if we want to exit process on errors/warnings. That way the changes in package would be backward compatible.

authors: ketanSaxena lwille

repo: ketanSaxena/schema-validator

license: MIT


text:
updateRecord writes data not record to json

line 102 in db.py
filo.writeJson(data_temp, data)
Should be
filo.writeJson(data_temp, record)

Thanks you for pointing it out. I will take a look into code and fix it as soon as possible along with other issue you raised.

Hope this  commit fixes the above mentioned problems. Let me know if you ever find another bug.
New contributions from your side will always be welcomed.

repo: ketanip/dbjson


text:
Unable to deserialize HashSet

I'm unable to deserialize HashSets with Jil:
    public class HashSetTest
    {
        public HashSetTest() {
            Tags = new HashSet<string>();
        }

        public HashSet<string> Tags { get; set; }
    }

Exception:
  Jil.DeserializationExceptionExpected: {, null
     at _DynamicMethod1(TextReader, Int32)
     at Jil.JSON.Deserialize(TextReader reader, Options options) in c:\Users\kmontrose\Dropbox\jil\Jil\JSON.cs: line 972
     at Jil.JSON.Deserialize(String text, Options options) in c:\Users\kmontrose\Dropbox\jil\Jil\JSON.cs: line 997

This is by design, Jil will deserialize a list to anything that implements IList<T> which HashSet<T> does not.  Supported types are mentioned in the README.
As to why this is... JSON doesn't have a notion of a unique set of elements, and it's lists/arrays are ordered.  It would make me nervous to have Jil change the order of elements in an array when serializing a deserialized bit of JSON.

This prevents us from ever using this library... HashSet implements ISet, and ISet is unordered, it just can't have duplicate values in it.. ( http://msdn.microsoft.com/en-us/library/dd412081(v=vs.110).aspx ). A sorted set will handle sorting the data as you put into it.. So deserializing that without worrying shouldn't be a problem..

If you are worried about duplicate data then let the runtime throw the error. If the user wants to put garbage in, he can get an error out.

JSON doesn’t have a notion of uniqueness, so it’s logically out of scope for a deserializer.
Implementing a ToHashSet on the caller side is trivial: http://stackoverflow.com/questions/3471899/how-to-convert-linq-results-to-hashset-or-hashedset

Yeah, I understand what your saying.. But why not try to deserialize the array to it and if it blows up for non uniqueness than its not your (Jils) problem but the user for giving it garbage. For example I want to use jil to serialize my object that has a hashset and put it into redis cache.. Getting it out I know its unique.

I think from a design perspective (not to put words in Kevin’s mouth), it becomes Jil’s problem because this would be asking it to do something more heuristic that deterministic. It introduces a new class of error.
A JSON deserializer should handle things that have clear representations or analogs in JSON. Hashsets do not. So, let the caller decide.
Jil’s handling of dates is analogous. It requires the caller specify exactly what date format to understand – Jil doesn’t get into “trying” to figure it out. Too heuristic. If one wants to parse dates heuristically, the caller can implement their own logic.

I don't mean to be a PITA... I guess I don't understand why this is such a problem (even though I understand what you are saying). Why do you support ICollection or IList? Shouldn't you only support T[]? My thought is you would just to do the conversion and under the hood it would call TryConvert on the type or something?
By the way a HashSet of String is serialized as ["My", "String", "Array"]

@niemyjski fyi, this is now supported via #219 and released as 2.14.0.

Thank you!!!!

authors: bretcope clipperhouse kevin-montrose niemyjski

repo: kevin-montrose/Jil

license: mit


text:
Doesn't play nicely with requests-ntlm

Here's what happens when you use auth=requests_ntlm.HttpNtlmAuth(username, password):
  File "/venv/lib/python3.5/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/venv/lib/python3.5/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/venv/lib/python3.5/site-packages/requests/sessions.py", line 512, in request
    resp = self.send(prep, **send_kwargs)
  File "/venv/lib/python3.5/site-packages/requests/sessions.py", line 629, in send
    r = dispatch_hook('response', hooks, r, **kwargs)
  File "/venv/lib/python3.5/site-packages/requests/hooks.py", line 31, in dispatch_hook
    _hook_data = hook(hook_data, **kwargs)
  File "/venv/lib/python3.5/site-packages/requests_ntlm/requests_ntlm.py", line 151, in response_hook
    kwargs
  File "/venv/lib/python3.5/site-packages/requests_ntlm/requests_ntlm.py", line 52, in retry_using_http_NTLM_auth
    server_certificate_hash = self._get_server_cert(response)
  File "/venv/lib/python3.5/site-packages/requests_ntlm/requests_ntlm.py", line 187, in _get_server_cert
    socket = raw_response._fp.fp.raw._sock
AttributeError: 'VCRHTTPResponse' object has no attribute 'fp'

A lot of changes have happened to VCRpy since this ticket was opened. As this ticket has become stale, would you mind closing it if it is no longer needed / relevant?
If I haven't heard anything in a week I'll mark it as closed as we have a lot of old tickets that need to be groomed to make it easier to see what is still relevant.
However if it is still needed, please feel free to re-open or create a new ticket.
Thanks! 🙏

I'm having a similar issue with NTLM authentication.  I've managed to come up with a small pytest script that uses a public NTLM test webpage that shows the issue.
import requests
import requests_ntlm
import vcr

def test_ntlm_auth():
    resp = requests.get(
        "http://ntlm.herokuapp.com/",
        auth=requests_ntlm.HttpNtlmAuth("user", "pass"),
    )
    assert "ello" in str(resp.content)

def test_ntlm_auth_with_vcr():
    with vcr.use_cassette("/tmp/test_ntlm_auth.yaml"):
        resp = requests.get(
            "http://ntlm.herokuapp.com/",
            auth=requests_ntlm.HttpNtlmAuth("user", "pass"),
        )
        assert True

Output running with pytest -v looks like this:
$ pytest -v
================================================= test session starts ==================================================
platform linux -- Python 3.8.6, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /home/marko/.pyenv/versions/3.8.6/envs/ntlm_auth/bin/python3.8
cachedir: .pytest_cache
rootdir: /mnt/c/Users/Mark Osbourne/ntlm_auth
collected 2 items

test_ntlm_auth.py::test_ntlm_auth PASSED                                                                         [ 50%]
test_ntlm_auth.py::test_ntlm_auth_with_vcr FAILED                                                                [100%]

======================================================= FAILURES =======================================================
_______________________________________________ test_ntlm_auth_with_vcr ________________________________________________

    def test_ntlm_auth_with_vcr():
        with vcr.use_cassette("/tmp/test_ntlm_auth.yaml"):
>           resp = requests.get(
                "http://ntlm.herokuapp.com/",
                auth=requests_ntlm.HttpNtlmAuth("user", "pass"),
            )

test_ntlm_auth.py:16:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/marko/.pyenv/versions/3.8.6/envs/ntlm_auth/lib/python3.8/site-packages/requests/api.py:76: in get
    return request('get', url, params=params, **kwargs)
/home/marko/.pyenv/versions/3.8.6/envs/ntlm_auth/lib/python3.8/site-packages/requests/api.py:61: in request
    return session.request(method=method, url=url, **kwargs)
/home/marko/.pyenv/versions/3.8.6/envs/ntlm_auth/lib/python3.8/site-packages/requests/sessions.py:542: in request
    resp = self.send(prep, **send_kwargs)
/home/marko/.pyenv/versions/3.8.6/envs/ntlm_auth/lib/python3.8/site-packages/requests/sessions.py:662: in send
    r = dispatch_hook('response', hooks, r, **kwargs)
/home/marko/.pyenv/versions/3.8.6/envs/ntlm_auth/lib/python3.8/site-packages/requests/hooks.py:31: in dispatch_hook
    _hook_data = hook(hook_data, **kwargs)
/home/marko/.pyenv/versions/3.8.6/envs/ntlm_auth/lib/python3.8/site-packages/requests_ntlm/requests_ntlm.py:146: in response_hook
    return self.retry_using_http_NTLM_auth(
/home/marko/.pyenv/versions/3.8.6/envs/ntlm_auth/lib/python3.8/site-packages/requests_ntlm/requests_ntlm.py:52: in retry_using_http_NTLM_auth
    server_certificate_hash = self._get_server_cert(response)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <requests_ntlm.requests_ntlm.HttpNtlmAuth object at 0x7f6bed88c5e0>, response = <Response [401]>

    def _get_server_cert(self, response):
        """
        Get the certificate at the request_url and return it as a hash. Will get the raw socket from the
        original response from the server. This socket is then checked if it is an SSL socket and then used to
        get the hash of the certificate. The certificate hash is then used with NTLMv2 authentication for
        Channel Binding Tokens support. If the raw object is not a urllib3 HTTPReponse (default with requests)
        then no certificate will be returned.

        :param response: The original 401 response from the server
        :return: The hash of the DER encoded certificate at the request_url or None if not a HTTPS endpoint
        """
        if self.send_cbt:
            certificate_hash = None
            raw_response = response.raw

            if isinstance(raw_response, HTTPResponse):
                if sys.version_info > (3, 0):
>                   socket = raw_response._fp.fp.raw._sock
E                   AttributeError: 'NoneType' object has no attribute 'raw'

/home/marko/.pyenv/versions/3.8.6/envs/ntlm_auth/lib/python3.8/site-packages/requests_ntlm/requests_ntlm.py:187: AttributeError
=============================================== short test summary info ================================================
FAILED test_ntlm_auth.py::test_ntlm_auth_with_vcr - AttributeError: 'NoneType' object has no attribute 'raw'
============================================= 1 failed, 1 passed in 0.71s ==============================================

I found a workaround for this issue that works in my particular situation.
I use pytest, in conftest.py, I add the following monkeypatch:
from vcr.stubs import VCRHTTPResponse
from unittest import mock

def stubbed__setattr__(self, attribute, value):
    if attribute == 'fp':
        value = mock.Mock(raw=mock.Mock(_sock=None))
    rez = super(VCRHTTPResponse, self).__setattr__(attribute, value)
    return rez

VCRHTTPResponse.__setattr__ = stubbed__setattr__

authors: benwah ghost gotmarko neozenith


text:
[WIP] Game Model Revamp

It's time to add logic to the Game model for:

[ ] game type / hammer order (palms, normal, doubles, singles)
[ ] number of frames
[ ] allow ties
[ ] switch color after x frames

In the past, we had this scattered around the views and controllers, as this was only used for the live score overlays.

Forgot about Draft PRs

repo: kevinreedy/shuffio


text:
🛑 FX blog (kty) is down

In 72f15c8, FX blog (kty) (https://foxter-blogeditor.konecty.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX blog (kty) is back up in b1cde98 after 9 minutes.

repo: keviocastro/upptime


text:
🛑 FX blog (kty) is down

In a016cba, FX blog (kty) (https://foxter-blogeditor.konecty.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX blog (kty) is back up in f35b3b2 after 6 minutes.


text:
🛑 FX blog (kty) is down

In 11eb7da, FX blog (kty) (https://foxter-blogeditor.konecty.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX blog (kty) is back up in f908076 after 8 minutes.


text:
🛑 sinicesp site (kty) is down

In 3ace5fd, sinicesp site (kty) (http://conexaopcd.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: sinicesp site (kty) is back up in 543fcf1.


text:
🛑 FX corretor (kty) is down

In 52b0452, FX corretor (kty) (https://corretores.foxterciaimobiliaria.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX corretor (kty) is back up in f761469 after 6 minutes.


text:
🛑 FX corretor (kty) is down

In a66e647, FX corretor (kty) (https://corretores.foxterciaimobiliaria.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX corretor (kty) is back up in a0bd4bc after 6 minutes.


text:
🛑 FX corretor (kty) is down

In c587a87, FX corretor (kty) (https://corretores.foxterciaimobiliaria.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX corretor (kty) is back up in acec102 after 8 minutes.


text:
🛑 FX corretor (kty) is down

In 8db50f6, FX corretor (kty) (https://corretores.foxterciaimobiliaria.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX corretor (kty) is back up in 7caa07e after 6 minutes.


text:
🛑 FX corretor (kty) is down

In 0ea1705, FX corretor (kty) (https://corretores.foxterciaimobiliaria.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX corretor (kty) is back up in 684e8e4 after 26 minutes.


text:
🛑 FX blog (kty) is down

In fdb2dc1, FX blog (kty) (https://foxter-blogeditor.konecty.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX blog (kty) is back up in 20ff662 after 6 minutes.


text:
🛑 FX corretor (kty) is down

In 01f3b5b, FX corretor (kty) (https://corretores.foxterciaimobiliaria.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX corretor (kty) is back up in 05221ab after 35 minutes.


text:
🛑 urba integration (kty) is down

In 81e50c8, urba integration (kty) (https://urbacon-integrations.konecty.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: urba integration (kty) is back up in 763d0e2.


text:
🛑 FX blog (kty) is down

In ad7bb30, FX blog (kty) (https://foxter-blogeditor.konecty.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX blog (kty) is back up in 639c0fe after 6 minutes.


text:
🛑 FX corretor (kty) is down

In 249a3b3, FX corretor (kty) (https://corretores.foxterciaimobiliaria.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX corretor (kty) is back up in 9f350ea after 11 minutes.


text:
🛑 FX blog (kty) is down

In 92de953, FX blog (kty) (https://foxter-blogeditor.konecty.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX blog (kty) is back up in 7827cd5 after 10 minutes.


text:
🛑 FX corretor (kty) is down

In b4084e3, FX corretor (kty) (https://corretores.foxterciaimobiliaria.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX corretor (kty) is back up in 4f37a7a after 10 minutes.


text:
🛑 FX blog (kty) is down

In bd01602, FX blog (kty) (https://foxter-blogeditor.konecty.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX blog (kty) is back up in 9dec94f after 35 minutes.


text:
🛑 FX corretor (kty) is down

In d476f6b, FX corretor (kty) (https://corretores.foxterciaimobiliaria.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX corretor (kty) is back up in 754f93c after 26 minutes.


text:
🛑 FX blog (kty) is down

In 8e48db0, FX blog (kty) (https://foxter-blogeditor.konecty.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX blog (kty) is back up in 36f51cd after 6 minutes.


text:
🛑 FX net (kty) is down

In 6148b67, FX net (kty) (https://foxter.net.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX net (kty) is back up in 0d4cb2c.


text:
🛑 FX corretor (kty) is down

In 68501c8, FX corretor (kty) (https://corretores.foxterciaimobiliaria.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX corretor (kty) is back up in 52abcb1 after 6 minutes.


text:
🛑 FX blog (kty) is down

In 8fb46f2, FX blog (kty) (https://foxter-blogeditor.konecty.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX blog (kty) is back up in 95b0564 after 42 minutes.


text:
🛑 FX corretor (kty) is down

In 369e59d, FX corretor (kty) (https://corretores.foxterciaimobiliaria.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX corretor (kty) is back up in 7761cd0 after 46 minutes.


text:
🛑 FX blog (kty) is down

In 0767e85, FX blog (kty) (https://foxter-blogeditor.konecty.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX blog (kty) is back up in 9fb4ab4 after 6 minutes.


text:
🛑 AVS rgsp (kty) is down

In 4820d1e, AVS rgsp (kty) (https://rgsp.avus.com.br) was down:

HTTP code: 404
Response time: 193 ms

Resolved: AVS rgsp (kty) is back up in fdf5260.


text:
🛑 FX blog (kty) is down

In e3930fe, FX blog (kty) (https://foxter-blogeditor.konecty.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FX blog (kty) is back up in f3a3a96 after 7 minutes.


text:
Added new site. Updated sites.yml

Added forgetful.dev to the sites list.

This PR is:

[x] Adding a new domain
[ ] Updating existing domain size
[ ] Changing domain name
[ ] Removing existing domain from list
[ ] Website code changes (512kb.club site)
[ ] Other not listed

[x] I used the uncompressed size of the site
[x] I have included a link to the GTMetrix report
[x] The domain is in the correct alphabetical order
[x] This site is not a ultra lightweight site
[x] The following information is filled identical to the data file

I confirm that I have read the FAQ section, particularly the two red items around minimal pages and inappropriate content, and I attest that this site is neither of these things.

[x] Check to confirm

- domain: forgetful.dev
  url: https://forgetful.dev/
  size: 74.3
  last_checked: 2023-06-12

GTMetrix Report: https://gtmetrix.com/reports/forgetful.dev/7d6771gm/

Thanks!

authors: datastring garritfra


text:
Remove ecliptik.com from showcase - no longer using simple.css

@kevquirk just an fyi that I'm no longer using simple.css on my personal site ecliptik.com which is featured on https://simplecss.org/showcase.
I really enjoyed the theme and it was nice having it on the showcase, but this week I moved over to using a different theme (console.css) since I liked the terminal/gemini type look for the page.
Thank you for making simple.css, it's a wonderful framework to use.

Thanks for letting me know - that is a cool looking theme!

authors: ecliptik kevquirk


text:
componentWillReceiveProps changes 1

Updating componentWillReceiveProps: More complicated than the didMount change.
See patterns here: https://reactjs.org/blog/2018/03/27/update-on-async-rendering.html

@keybase/react-hackers this is part of of updating componentWillReceiveProps. There are a couple of patterns that are based on the blog post above. Sometimes the intention is to just make some state off of props. sometimes we want to make a call, etc

note: i'm marking the folders stuff as unsafe as we're going to deprecate that whole section before any of this matters

@keybase/react-hackers bump

@keybase/react-hackers bump

@buoyad
https://github.com/facebook/react/issues/12516

they have a pr for hot loader: https://github.com/gaearon/react-hot-loader/pull/927

@chrisnojima ok cool!

authors: buoyad chrisnojima

repo: keybase/client


text:
Use custom icons in for exploding messages in inbox

Description
Currently, we use the emojis 💥 and 💣 to signify exploded and soon-to-explode messages in small teams in a user's inbox. We use custom icons everywhere else, so this update replaces these emojis with those custom icons.
Solution
It's actually defined in the API to return these emojis, so I wrote a switch statement to convert these emojis (or the lack of one) into the appropriate Icon component. There was also some difficulty centering the Icon components, so I placed them in a Box2.
I also added a couple more conversations with the new icons to the simple inbox story in Storybook and updated our snapshot accordingly.

@keybase/react-hackers

@keybase/react-hackers could probably use a bump if anyone has a few minutes. Lemme know merging strategy with #12718, too.

Also updated the look a little bit to better reflect @cecileboucheron's designs. Here's what my inbox looks like now:

@cecileboucheron, should those boom!s be gray instead of black?

Yeah agreed, let's do black_40 for the boom icon. Also the bomb could be a bit smaller I think - maybe 12px font-size on desktop and 16 on mobile, since the snippet is using body small.

Updated!

LGTM!


text:
Focus reactji emoji picker on button press

The autoFocus prop doesn't seem to work, although it works for the
emoji button in the chat input box.

Okay, I fixed it so that ReactDOM is imported on desktop only. PTAL!

fixed circleci


text:
Thunderbolt icon color folder DESKTOP-1090

Now the thunderbolt icon appropriately matches the text/desired color. This takes advantage of the new font icons implemented by @MarcoPolo.
Look at these sweet examples of color matching:

@keybase/react-hackers

@chromakode any idea who no visdiff?

@chrisnojima the build failed on flow before visdiff would run due to this weekend's flow breakage.

:mag_right: These commits introduced some visual changes:

Changed: Menubar-Normal (before) (after)

Changed: Folders_TLF-Normal (before) (after)

Changed: Files-Popup_Public (before) (after)

Changed: Files-Popup_Private (before) (after)

Changed: Files-Normal_Public (before) (after)

Changed: Files-Normal_Private (before) (after)

Changed: File-Normal_Public (before) (after)

Changed: File-Normal_Private_modified_by_self_long_file (before) (after) (diff)
Changed: File-Normal_Private_Long (before) (after) (diff)
Changed: File-Normal_Private (before) (after) (diff)

👍 i'm fine with the small offsets, we can fix that up later if design is annoyed

authors: awendland chrisnojima chromakode keybase-ci-visdiff


text:
CreateFile not call when file name contains colon?

On Windows, CreateFile can call when file name contains *?<>|", but CreateFile not call when file name contains colon, why?

Windows doesn't allow colons in filenames: https://www.quora.com/Why-are-colons-not-allowed-in-the-filenames-of-Windows-files

@strib ,yes, Windows doesn't allow colons in filenames, but dokan allow colons in filename, ZwCreateFile function can callback when i use dokan c lib, ZwCreateFile function can not callback when i use kbfs

I assume we don't allow it so as not to confuse certain Windows applications.  @taruti might remember more.

Allowing files that won't work with common applications could lead to problems. Especially in shared folders with multiple people.
@barcelona-messi what kind of use-cases do you have? E.g. for maildirs usually an alternate separator character is used on Windows based systems.

@taruti , file name allow contain colons in iOS or linux, i want mount iOS  or linux filesystem

authors: barcelona-messi strib taruti

repo: keybase/kbfs


text:
tlf_journal: check MD server for conflicts during flush

If we have a lot of blocks to flush from the journal, we should check
the MD server periodically to see if the next MD we're going to flush
would result in a conflict.
Specifically, if we have already once done CR locally, we might have a
huge set of blocks to flush, and only one MD to flush at the very end.
If we wait to the end to find out if there's a conflict, it could take
a long time for the remote changes (written by another device) to be
reflected in the local view of the TLF.  If instead we force CR
earlier, those changes will become visible earlier.
Issue: KBFS-2106

Thanks for the feedback, implemented the suggestions I didn't comment on directly.  Will merge after CI passes.


text:
Have a single module with synced classes from keycloak/keycloak

Closes #34

Not 100% sure if the name of the maven module name can be only keycloak-client-common as the fact that sources are synced might be an "implementation detail" from the point of view of the consumer of that maven artifact? But does not matter much to me, so approving.

I just wanted to stress that this module is purely synced and should not be used directly by anybody.

Not 100% sure if the name of the maven module name can be only keycloak-client-common as the fact that sources are synced might be an "implementation detail" from the point of view of the consumer of that maven artifact? But does not matter much to me, so approving.

I just wanted to stress that this module is purely synced and should not be used directly by anybody.

That works for me. Thanks for the clarification!

authors: mposolda pskopek

repo: keycloak/keycloak-client


text:
Resolve the issue: Some more dev env stubs to clean up

Delete redundant code in  config.py

Incorrect information submitted

repo: keylime/keylime


text:
No option to use original filename

It looks like the generateFilename function is set to nameFunctions.randomFilename - meaning any upload using this storage adapter will always have a random name.
Suggest adding an option to allow the original name to be used instead.
Concept:
    if(options.s3.originalname) {
        this.options.generateFilename = function(file, attempt, callback) {
            return callback(null, file.originalname);
        }
    } else {
        this.options.generateFilename = ensureCallback(this.options.generateFilename);
    }

Or should this be handled somewhere higher up in the keystone Types? Or be changed to support a filename function in the Field definition (e.g., filename: (file) => file.originalName)?

Looking at the forks it seems like @GwonHyeok already found a fix for the issue.

CC: @JedWatson

For anyone that came here looking for an example of how to use the original file name with this plugin, check out https://gist.github.com/robksawyer/6060412c3f86ddbc68b29ed09eef75c3.

@robksawyer thank you - that worked a treat 🎉

Just wanted to note that I was having difficulty in s3 with files that have spaces in them, so when I want to keep the original file name I use this:
generateFilename: file => file.originalname.replace(/ /g, '-'),

I'll also mention this for anyone who wants to generate a public url that is saved. I usually use a method that looks like this:
publicUrl : file => `https:${process.env.S3_BASE_URL}${s3FilePath}${file.filename}`,

The S3_BASE_URL is saved in the .env file. Instead you may have a cloudfront path or whatever else you want.

authors: internetErik mhoffmeyerDC n-devr robksawyer


text:
🛑 OP Financial Group is down

In b8b5e40, OP Financial Group (https://www.op.fi/home-page) was down:

HTTP code: 0
Response time: 0 ms

Resolved: OP Financial Group is back up in 3b56d8b after 7 minutes.

repo: keywordnew/vantaa-watchtower


text:
ugly token output

before
Search results:

Solution 1:
Solution:Map -->
X:Term |-> john (  )

after
Search results:

Solution 1:
Solution:Map -->
#token ( X:Term , MetaVariable ) |-> #token ( john , Literal )

looking at KOREToTreeNodes, line 30-32, it seems this happens when printing tokens which do not have an associated production in the definition given to unparse. It's likely the module structure is strange in some regard.

"Strange" = the semantics module does not have the productions for MetaVariable and Literal

@grosu, is this verbose output acceptable for now?

Fine, although it obviously needs to change soon after the release.  We don't want students to see this.  So fine to modify the .out files, but please add a NOTES.md and explain it and say that we are going to make it look good soon.

@laurayuwen, before you do that, I'll attempt a quick fix.

#2088 should fix this for now.

@laurayuwen, give it a try and merge if it works.

authors: cos grosu laurayuwen

repo: kframework/k


text:
Remove trivial branches when simplifying claims at startup

I'm debugging the https://github.com/kframework/wasm-semantics/blob/memory-spec/tests/proofs/memory-spec.k under memory-spec branch in wasm-semantics.
I found that in 0449856d84f82eea8df31f8510ed5ed1066cdb36, after simplifying the LHS, it split the original claim into 2. One of them is
  ( #Ceil ( ADDR modInt 4294967296 )
    #And
      {
        EA
      #Equals
        ADDR modInt 4294967296
      }
  #Implies
    {
      true
    #Equals
      EA +Int #numBytes ( ITYPE ) <=Int SIZE *Int 65536 andBool #isByteMap ( ByteMap <| BM0 |> )
    } )
#And
  #Ceil ( ADDR modInt 4294967296 )
#And
  #Not ( #Ceil ( ADDR modInt 4294967296 )
  #And
    #Not ( #Ceil ( ADDR modInt 4294967296 )
    #And
      {
        EA
      #Equals
        ADDR modInt 4294967296
      } ) )
#And
  #Not ( #Ceil ( ADDR modInt 4294967296 )
  #And
    {
      EA
    #Equals
      ADDR modInt 4294967296
    } )

#Ceil (... modInt ...) is simplifiable.
We should simplify the predicate to remove trivial branches of the proof.

We saw similar not(a) #and not(not(a)) in #1060

Duplicate of #1103.

I think 1. is fixed on the latest kore master, could you please double check?

Yes, the Ceil(... modInt ...) problem is definitely fixed.

authors: ana-pantilie ttuegel yzhang90

repo: kframework/kore


text:
kore-exec: Use compacting garbage collector on oldest branch

This is a work-around for #1335. It doesn't solve the underlying problem, but it
should reduce memory use in the meantime.

Reviewer checklist

[ ] Test coverage: stack test --coverage
[ ] Public API documentation: stack haddock

The fix is not that promising...The kore-repl still eats most of the memory after executing 22 steps...

I still need to call graph  every 11 steps... ( it seems that 15 - 18 steps is the limit on my computer)


text:
amalgkit config

amalgkit metadata should be started more easily, and there should be a single command that generates all necessary config files (near empty to be neutral enough for any purposes/organisms). A new subcommand should look like csubst dataset and may be named amalgkit config. @Hego-CCTB Could you implement it?

On it!

amalgkit config is done:
https://github.com/kfuku52/amalgkit/commit/dd20107aa74597a11d324a767c804cb12184ec47
Had to hard-code the file creation for all .config files, since I wanted to keep the headers explaining how to use the config files.
Currently there are three parameters:
--out_dir, default: ./
directory where config will use as working_directory (will create a folder named config in --out_dir)
--config_dir, default empty_config
name of the folder where the config files will be created in
--overwrite, default no
if --config_dir already exists, amalgkit config will stop immediately if --overwrite is no. If yes, config files will be overwritten insetead.
Resulting directory infrastructure:
out_dir <-- can be changed
│
└───config <-- can not be changed
│   │
│   └───config_dir <-- can be changed
│       │ control_term.config
│       │ exclude_id.config
│       │   ...

Thanks. Can we stick to the way used in csubst dataset (i.e., just copy files to a specified directory)? The config files should be near-empty, but not completely empty because there are commonly applied conditions, such as "Filter"	"type rnaseq" in search_term_other.config, and it is difficult to modify if hardcoded in the python script. I see no problem with headers, so could you explain it more?

Ah, now the csubst code makes more sense to me. There is a nearly empty set of files somewhere in the installation folder (i.e. manually created), and the command just copies them somewhere else?
Whereas I created the files from scratch.

Yes, the files for csubst dataset are placed here: https://github.com/kfuku52/csubst/tree/master/csubst/dataset

alright, took me a while to figure out how this works, but here's the new version:
https://github.com/kfuku52/amalgkit/commit/0fe5bb77ae764ced3724428bb94a900a307b2261
At first I wanted to use pkg_resources as in csubst, but upon further investigation it is apparently better to use importlib.resources to retrieve files from within the package, since pkg_resources is deprecated now.
Anyways, how amalgkit config works now is the following:
--overwrite terminates the process if the target directory is already there and set to no.
--config_dir this is the name of the config directory to be created. i.e. the destination (defaults to the name of --config)
--config this is the name of the config directory stored in the amalgkit package. i.e. the source
currently there are four options: 'test', 'plantae', 'vertebrate' and 'base'. 'base' is the nearly empty dataset, where I left just the header and 1-3 lines as an example within each of the config files.
--out_dir working directory
Example:
amalgkit config --config base --config_dir my_config --out_dir  ./

will create ./config/my_config/ and copy all .config files in there.
amalgkit config --config vertebrate --out_dir  ./

will create ./config/vertebrate/ and copy all .config files in there.
Note: I had to make copy the config folder inside amalgkit and give it a different name for this to work. The config folder is now redundant, so we can delete it.

I pushed another update, which should fix this issue.
Can you try it out on your end?
https://github.com/kfuku52/amalgkit/commit/eb22ce5cc382472079b45cbfa8669063c869a2b6

The latest version worked well. Thank you!

Great! I will close this for now, then.


text:
Research : This tool is probably not that functional

Maybe I'm wrong
I heard about this tool earlier, and I would like to know if this tool can be used in order to obtain the correct cookies/token to get to the YouTube channel, but what I did research and read about it is that it is not possible to use classic cookies for bypass security protocols on youtube
Questions
- Does it work on youtube too?
- It also bypasses TSL and HTTP? Because these things are responsible for prevention against MITM attacks.
-Do you like cats?
-MITM attacks require you both to be in the same network,how this tool work than
thankq you for any help, if you have any additional information I would be very happy for it

Yes
Yes
No
No, This is more comparable to phishing

@AntiChrist-Coder
okay, I would like to ask you this

why don't you like cats
Will this repository work on evginx3? I assume that maybe the synstasis was changed when switching from version 2 to 3, it is also very old and cookies handling and session handling can be completely different
Link : https://github.com/An0nUD4Y/Evilginx2-Phishlets/blob/master/google.yaml
3.Google uses obfuscated javascript to identify the login as coming from a phishing page. You gotta deobfuscate it and then find a way to bypass it. So it could be a problem.

It is functional and a great tool :)

@rickyrik001  wher find gmail phshlet?

@devill11100 I told you how you can do it,if you need further assistance contact me on telegram.@Ricky_rik1


text:
🛑 Fvideo is down

In fda902e, Fvideo (https://www.fvideo.ro) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Fvideo is back up in 74e0a64 after 10 minutes.


text:
[Snyk] Security upgrade numpy from 1.21.3 to 1.22.2

Snyk has created this PR to fix 3 vulnerabilities in the pip dependencies of this project.
Snyk changed the following file(s):

examples/heatmap_and_track/requirements.txt

⚠ Warning
ultralytics 8.0.151 requires scipy, which is not installed.
ultralytics 8.0.151 has requirement numpy>=1.22.2, but you have numpy 1.21.3.

[!IMPORTANT]

Check the changes in this PR to ensure they won't cause issues with your project.
Max score is 1000. Note that the real score may have changed since the PR was raised.
This PR was automatically created by Snyk using the credentials of a real user.
Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
📜 Customise PR templates
🛠 Adjust project settings
📚 Read about Snyk's upgrade logic

Learn how to fix vulnerabilities with free interactive lessons:
🦉 NULL Pointer Dereference
Summary by Sourcery
Bug Fixes:

Fixed security vulnerabilities by upgrading numpy.

:tada: Snyk checks have passed. No issues have been found so far.
:white_check_mark: security/snyk check is complete. No issues have been found. (View Details)

repo: khulnasoft/superverse


text:
[BUG]: Misalignment of Send Message button in Footer of Menu page

The Send Message Button in Contact Us section in Footer of Menu page is not appropriately aligned.
Here's a screenshot,

Hello @khushi-joshi-05 and @sunny0625
Please assign me this issue under GSSOC'24
Thank You!

Hello @sunny0625  and @khushi-joshi-05
Please assign me this issue under GSSOC'24.


text:
Remove BUILDPACK_URL

This isn't required because we officially support Go now: https://blog.heroku.com/archives/2015/7/7/go_support_now_official_on_heroku

Thanks for this!
Sorry for the late answer, this is a pretty old repo I just came back to for a little refresh.

authors: freeformz kiasaki

repo: kiasaki/marks


text:
Force ReRender When .map Completes

Hello,
I am trying to use the swiper by populating it with a variable number of components based on an array. However, it seems the swiper needs to be rerendered after the map completes.
const colorParams = {
  slidesPerView: 3,
  spaceBetween: 30,
  freeMode: true,
  rebuildOnUpdate: true
};

<Swiper {...colorParams}>
  {
    ColorOptions.map((color) => {
      return (
        <ColorCircle /> /* puedo component code */
      );
   })
 }
</Swiper>

I don't see anything in the docs about this and am hoping I'm just missing something. All of my other testing looks to work fine up until I try to populate the slider with a map method.

Hi @yuschick , u can use params rebuildOnUpdate: true or shouldSwiperUpdate : true.  If u wanna rebuild Swiper each time component got re-rendered , just use rebuildOnUpdate. If u wanna update Swiper only, just use shouldSwiperUpdate

I have the same issue and not working with shouldSwiperUpdate  and rebuildOnUpdate .

such a beautiful tool but doesnt work with map well, at tried above methods rebuildOnUpdate, at first run it doesnt work but if you navigate to any other page with router and come back it works, dont know how to fix it .. wish it worked well with fetched data from db via redux store .... 👎

worked like a charm i was using it in params but should be used like below
<Swiper {...params} shouldSwiperUpdate>

It's not working!

In your object colorParams include the following propertie
rebuildOnUpdate: true

Neither props are recognized by react...
React does not recognize the rebuildOnUpdate prop
React does not recognize the shouldSwiperUpdate prop

I've tested them today and rebuildOnUpdate + shouldSwiperUpdate still don't work in react. We need something like this for some maps with more logic.

I've tested them today and rebuildOnUpdate + shouldSwiperUpdate still don't work in react. We need something like this for some maps with more logic.

hey, any updates ?

authors: Anonymous-dev-v CathyMacars configbug kidjp85 kirsantov malikgenius omidnikrah osklars yuschick


text:
fixed parameter definition and removed timeout

have to use double square bracket for variable definition and removed time outs in builds as they are to small depending the traffic on nodes

Jenkins please retest this

repo: kiegroup/kie-jenkins-scripts


text:
[1.9.x] KOGITO-5525 Add protobuf marshalling related resources to native image

Assembly:

https://github.com/kiegroup/kogito-runtimes/pull/1461
https://github.com/kiegroup/kogito-examples/pull/804

The (build) Apps check has failed. Please check the logs.

jenkins rerun apps tests

please wait for the release 1.9.0 to be over before merging this one and its dependent ones

authors: cristianonicolai kie-ci radtriste

repo: kiegroup/kogito-runtimes