text:
Scoring two string inputs -- feature request and attempted implementation

I am trying to use an existing sockeye model (the monolingual ParaBank model from JHU and https://github.com/decompositional-semantics-initiative/ParaBank-rewriter) to score example input/output pairs that I have from a separate model and python/library. Due to constraints with this other program, it's a little too expensive to write each input/output pair to file, which is what is required when using the standard machinery in score.py (https://github.com/awslabs/sockeye/blob/master/sockeye/score.py).
Below is my attempt to rewrite score.py (let's call it score_string.py) that works from string input. When I run the following:
python score_string.py --model ~/third_party/ParaBank-rewriter/ --use-cpu --target tgt.txt --source src.txt --batch-size 1 (--target and --source are empty here for now).
I get the following output and error (among a few others):
[INFO:sockeye.utils] Sockeye version 1.18.85, commit c1b1da8dc154a87cde0c41476eac2c7633fe6f9a, path /Users/kyler/miniconda2/envs/rewrite_logic/lib/python3.6/site-packages/sockeye/__init__.py
[INFO:sockeye.utils] MXNet version 1.3.1, path /Users/kyler/miniconda2/envs/rewrite_logic/lib/python3.6/site-packages/mxnet/__init__.py
[INFO:sockeye.utils] Command: /Users/kyler/projects/rewrite_logic/rewrite_logic/sockeyeSeq2Seq.py --model /Users/kyler/third_party/ParaBank-rewriter/ --use-cpu --target tgt.txt --source src.txt --batch-size 1
[INFO:sockeye.utils] Arguments: Namespace(batch_size=1, batch_type='word', config=None, device_ids=[-1], disable_device_locking=False, length_penalty_alpha=1.0, length_penalty_beta=0.0, lock_dir='/tmp', loglevel='INFO', max_seq_len=None, model='/Users/kyler/third_party/ParaBank-rewriter/', num_words=(0, 0), output=None, output_type='score', pad_vocab_to_multiple_of=None, quiet=False, score_type='neglogprob', shared_vocab=False, softmax_temperature=None, source='src.txt', source_factor_vocabs=[], source_factors=[], source_vocab=None, target='tgt.txt', target_vocab=None, use_cpu=True, word_min_count=(1, 1))
[INFO:__main__] Scoring Device(s): cpu(0)
[INFO:sockeye.model] ModelConfig loaded from "/Users/kyler/third_party/ParaBank-rewriter/config"
[INFO:sockeye.vocab] Vocabulary (15989 words) loaded from "/Users/kyler/third_party/ParaBank-rewriter/vocab.src.0.json"
[INFO:sockeye.vocab] Vocabulary (13585 words) loaded from "/Users/kyler/third_party/ParaBank-rewriter/vocab.trg.0.json"
[INFO:sockeye.data_io] ==============================
[INFO:sockeye.data_io] Creating scoring data iterator
[INFO:sockeye.data_io] ==============================
[INFO:sockeye.model] Config[_frozen=True, config_data=Config[_frozen=True, data_statistics=Config[_frozen=True, average_len_target_per_bucket=[6.536779555828388, 12.695111308360453, 21.113829676281792, 29.702584587020255, 37.97110374449988, 46.07846054633612, 53.85288055852154, 61.082317040412825, 68.07096396771524, 74.83928896899513, 78.72540254824904, 81.3454750934538], buckets=[(9, 10), (18, 20), (27, 30), (36, 40), (45, 50), (54, 60), (63, 70), (72, 80), (81, 90), (90, 100), (99, 100), (100, 100)], length_ratio_mean=1.0645400645552539, length_ratio_std=0.29834278125977903, max_observed_len_source=100, max_observed_len_target=100, num_discarded=408226, num_sents=56657132, num_sents_per_bucket=[23211115, 18449333, 6644559, 3384680, 1940286, 1186265, 741384, 465335, 302562, 197516, 122594, 11503], num_tokens_source=881889667, num_tokens_target=869418202, num_unks_source=0, num_unks_target=0, size_vocab_source=15989, size_vocab_target=13585], max_seq_len_source=100, max_seq_len_target=100, num_source_factors=1, source_with_eos=True], config_decoder=Config[_frozen=True, attention_config=Config[_frozen=True, config_coverage=None, dtype=float32, input_previous_word=False, is_scaled=False, layer_normalization=False, num_heads=None, num_hidden=1024, query_num_hidden=1024, source_num_hidden=1024, type=dot], attention_in_upper_layers=False, context_gating=False, dtype=float32, enc_last_hidden_concat_to_embedding=False, hidden_dropout=0.2, layer_normalization=False, max_seq_len_source=100, rnn_config=Config[_frozen=True, cell_type=lstm, dropout_inputs=0.0, dropout_recurrent=0.0, dropout_states=0.0, dtype=float32, first_residual_layer=2, forget_bias=0.0, lhuc=False, num_hidden=1024, num_layers=6, residual=False], state_init=last, state_init_lhuc=False], config_embed_source=Config[_frozen=True, dropout=0.0, dtype=float32, factor_configs=None, num_embed=512, num_factors=1, source_factors_combine=concat, vocab_size=15989], config_embed_target=Config[_frozen=True, dropout=0.0, dtype=float32, factor_configs=None, num_embed=512, num_factors=1, source_factors_combine=concat, vocab_size=13585], config_encoder=Config[_frozen=True, conv_config=None, dtype=float32, reverse_input=False, rnn_config=Config[_frozen=True, cell_type=lstm, dropout_inputs=0.0, dropout_recurrent=0.0, dropout_states=0.0, dtype=float32, first_residual_layer=2, forget_bias=0.0, lhuc=False, num_hidden=1024, num_layers=6, residual=False]], config_loss=Config[_frozen=True, label_smoothing=0.1, name=cross-entropy, normalization_type=valid, vocab_size=13585], lhuc=False, vocab_source_size=15989, vocab_target_size=13585, weight_normalization=False, weight_tying=False, weight_tying_type=None]
[INFO:sockeye.encoder] sockeye.encoder.EncoderSequence dtype: float32
[INFO:sockeye.encoder] sockeye.encoder.ConvertLayout dtype: float32
[INFO:sockeye.encoder] sockeye.encoder.BiDirectionalRNNEncoder dtype: float32
[INFO:sockeye.encoder] sockeye.encoder.RecurrentEncoder dtype: float32
[INFO:sockeye.encoder] sockeye.encoder.RecurrentEncoder dtype: float32
[INFO:sockeye.encoder] sockeye.encoder.RecurrentEncoder dtype: float32
[INFO:sockeye.encoder] sockeye.encoder.ConvertLayout dtype: float32
[INFO:sockeye.decoder] sockeye.decoder.RecurrentDecoder dtype: float32
[INFO:sockeye.encoder] sockeye.encoder.Embedding dtype: float32
[INFO:sockeye.encoder] sockeye.encoder.Embedding dtype: float32
[INFO:sockeye.model] Loaded params from "/Users/kyler/third_party/ParaBank-rewriter/params.best"
[INFO:sockeye.data_io] Created bucketed parallel data set. Introduced padding: source=95.0% target=94.0%)

Segmentation fault: 11

Stack trace returned 10 entries:
[bt] (0) 0   libmxnet.so                         0x0000000113e2d740 libmxnet.so + 26432
[bt] (1) 1   libmxnet.so                         0x0000000115963136 MXTVMBridge + 4585942
[bt] (2) 2   libsystem_platform.dylib            0x00007fff5f237f5a _sigtramp + 26
[bt] (3) 3   libmxnet.so                         0x00000001154a8eae MXNDListFree + 1442638
[bt] (4) 4   libmxnet.so                         0x0000000113e624ab libmxnet.so + 242859
[bt] (5) 5   libmxnet.so                         0x0000000113e61f49 libmxnet.so + 241481
[bt] (6) 6   libmxnet.so                         0x000000011538d605 MXNDListFree + 281253
[bt] (7) 7   libmxnet.so                         0x000000011535b218 MXNDListFree + 75448
[bt] (8) 8   libmxnet.so                         0x000000011535e321 MXNDListFree + 88001
[bt] (9) 9   libmxnet.so                         0x000000011535e23f MXNDListFree + 87775

Below is the full script:
import math
import argparse
import logging
import os
import time
from contextlib import ExitStack
from typing import Optional, List, Tuple, Iterable
from sockeye import arguments
from sockeye import constants as C
from sockeye import data_io
from sockeye import inference
from sockeye import model
from sockeye import scoring
from sockeye import utils
from sockeye import vocab
from sockeye.log import setup_main_logger
from sockeye.output_handler import get_output_handler
from sockeye.utils import check_condition
from sockeye.scoring import ScoringModel,Scorer
from sockeye.data_io import (
    RawParallelDatasetLoader,
    BatchedRawParallelSampleIter,
    BaseParallelSampleIter,
    SequenceReader,
    BucketBatchSize,
    tokens2ids
)
from sockeye.output_handler import OutputHandler
from sockeye.inference import TranslatorInput,TranslatorOutput

logger = logging.getLogger(__name__)

class SimpleSequenceReader(Iterable):
    def __init__(self,sinput,vocabulary,add_eos=False,add_bos=False):
        self.sinput = sinput
        self.vocabulary = vocabulary
        self.add_eos = add_eos
        self.add_bos = add_bos

        assert C.UNK_SYMBOL in vocabulary
        assert vocabulary[C.PAD_SYMBOL] == C.PAD_ID
        assert C.BOS_SYMBOL in vocabulary
        assert C.EOS_SYMBOL in vocabulary
        self.bos_id = vocabulary[C.BOS_SYMBOL]
        self.eos_id = vocabulary[C.EOS_SYMBOL]

    def __iter__(self):
        sequence = tokens2ids(self.sinput,self.vocabulary)
        if self.add_bos:
            sequence.insert(0, self.bos_id)
        if self.add_eos:
            sequence.append(self.eos_id)
        yield sequence

def create_sequence_readers(sources: List[str], target: List[str],
                            vocab_sources: List[vocab.Vocab],
                            vocab_target: vocab.Vocab) -> Tuple[List[SequenceReader], SequenceReader]:
    """
    Create source readers with EOS and target readers with BOS.

    :param sources: The file names of source data and factors.
    :param target: The file name of the target data.
    :param vocab_sources: The source vocabularies.
    :param vocab_target: The target vocabularies.
    :return: The source sequence readers and the target reader.
    """
    source_sequence_readers = [SimpleSequenceReader(sources,vocab_sources[0],add_eos=True)] ## SEE ABOVE
    target_sequence_reader = SimpleSequenceReader(target,vocab_target,add_bos=True) ## SEE ABOVE
    return source_sequence_readers, target_sequence_reader

##################################################
# ## CUSTOM TYPE OF BATCHEDRAWPARALLELSAMPLEITER #
##################################################

class StringDataInput(BatchedRawParallelSampleIter):
    """A simplified version of the raw parallel sampler that simply takes two string inputs"""
    def __init__(self,data_loader,
                     sources,
                     target,
                     source_vocabs,
                     target_vocab,
                     bucket,
                     batch_size,
                     max_lens,
                     num_factors=1,
                     source_data_name=C.SOURCE_NAME,
                     target_data_name=C.TARGET_NAME,
                     label_name=C.TARGET_LABEL_NAME,
                     dtype='float32'
     ):
        BaseParallelSampleIter.__init__(self,buckets=[bucket], batch_size=batch_size, bucket_batch_sizes=[BucketBatchSize(bucket, batch_size, None)],
                         source_data_name=source_data_name, target_data_name=target_data_name,
                            label_name=label_name, num_factors=num_factors, permute=False, dtype=dtype)
        self.data_loader = data_loader
        self.sources_sentences,self.target_sentences =  create_sequence_readers(sources,target,source_vocabs,target_vocab) #SEE ABOVE
        self.sources_iters = [iter(s) for s in self.sources_sentences]
        self.target_iter = iter(self.target_sentences)
        self.max_len_source, self.max_len_target = max_lens
        self.next_batch = None
        self.sentno = 1

class SockeyeScorer(Scorer):
    """A seq2seq model based on sockeye"""
    def __init__(self,model: ScoringModel,
                 source_vocabs: List[vocab.Vocab],
                 target_vocab: vocab.Vocab) -> None:
        Scorer.__init__(self,model,source_vocabs,target_vocab)

        ## keeps the original vocabulary around to build data in string scorer
        self.orig_source_vocab = source_vocabs
        self.orig_target_vocab = target_vocab

    def string_scorer(self,src,target):
        """Redefined the scoring function

        :param src: the source text
        :param target: the target text
        """
        left_side = src.split()
        right_side = target.split()

        ## custom bucket size
        bucket = (100,100)

        #######################################
        # ON-THE-FLY DATA CREATE FROM STRINGS #
        #######################################

        data_loader = RawParallelDatasetLoader(buckets=[bucket],
                                           eos_id=self.orig_target_vocab[C.EOS_SYMBOL],
                                           pad_id=C.PAD_ID,
                                           skip_blanks=False)

        # special score iterator that builds from strings as opposed to from file

        score_iter = StringDataInput(data_loader=data_loader,
                                           sources=left_side,
                                           target=right_side,
                                           source_vocabs=self.orig_source_vocab,
                                           target_vocab=self.orig_target_vocab,
                                           bucket=bucket,
                                           batch_size=1,
                                           max_lens=(100,100),
                                           #num_factors=0
                                           )

        ## standard run through batch=1
        for batch_no,batch in enumerate(score_iter):

            ## attempt at scoring
            scores = self.model.run(batch)[0] ## raises seg fault and occasionally other errors

def get_data_iters_and_vocabs(args: argparse.Namespace,
                              model_folder: Optional[str]) -> Tuple['data_io.BaseParallelSampleIter',
                                                                    List[vocab.Vocab], vocab.Vocab, model.ModelConfig]:
    """
    Loads the data iterators and vocabularies.

    :param args: Arguments as returned by argparse.
    :param model_folder: Output folder.
    :return: The scoring data iterator as well as the source and target vocabularies.
    """

    ####################################################################
    # SAME AS IN ORIGINAL, IN THIS CONTEXT USED TO CREATE VOCABULARIES #
    ####################################################################

    model_config = model.SockeyeModel.load_config(os.path.join(args.model, C.CONFIG_NAME))

    if args.max_seq_len is None:
        max_seq_len_source = model_config.config_data.max_seq_len_source
        max_seq_len_target = model_config.config_data.max_seq_len_target
    else:
        max_seq_len_source, max_seq_len_target = args.max_seq_len

    batch_num_devices = 1 if args.use_cpu else sum(-di if di < 0 else 1 for di in args.device_ids)

    # Load the existing vocabs created when starting the training run.
    source_vocabs = vocab.load_source_vocabs(model_folder)
    target_vocab = vocab.load_target_vocab(model_folder)

    sources = [args.source] + args.source_factors
    sources = [str(os.path.abspath(source)) for source in sources]

    score_iter = data_io.get_scoring_data_iters(
        sources=sources,
        target=os.path.abspath(args.target),
        source_vocabs=source_vocabs,
        target_vocab=target_vocab,
        batch_size=args.batch_size,
        batch_num_devices=batch_num_devices,
        max_seq_len_source=max_seq_len_source,
        max_seq_len_target=max_seq_len_target)

    return score_iter, source_vocabs, target_vocab, model_config

def score(args: argparse.Namespace):

    setup_main_logger(file_logging=False, console=not args.quiet)

    utils.log_basic_info(args)

    with ExitStack() as exit_stack:

        ####################################
        # SAME AS IN ORIGINAL SCORE SCRIPT #
        ####################################

        context = utils.determine_context(device_ids=args.device_ids,
                                          use_cpu=args.use_cpu,
                                          disable_device_locking=args.disable_device_locking,
                                          lock_dir=args.lock_dir,
                                          exit_stack=exit_stack)
        if args.batch_type == C.BATCH_TYPE_SENTENCE:
            check_condition(args.batch_size % len(context) == 0, "When using multiple devices the batch size must be "
                                                                 "divisible by the number of devices. Choose a batch "
                                                                 "size that is a multiple of %d." % len(context))
        logger.info("Scoring Device(s): %s", ", ".join(str(c) for c in context))

        args.no_bucketing = True
        args.bucket_width = 10

        score_iter, source_vocabs, target_vocab, model_config = get_data_iters_and_vocabs(
            args=args,
            model_folder=args.model)

        scoring_model = ScoringModel(config=model_config,
                                             model_dir=args.model,
                                             context=context,
                                             provide_data=score_iter.provide_data,
                                             provide_label=score_iter.provide_label,
                                             default_bucket_key=score_iter.default_bucket_key,
                                             score_type=args.score_type,
                                             length_penalty=inference.LengthPenalty(alpha=args.length_penalty_alpha,
                                                                                beta=args.length_penalty_beta),
                                             softmax_temperature=args.softmax_temperature)

        ######################
        # NEW SCORING MODEL  #
        ######################

        ## call to new sockeye scorer model defined above
        scorer = SockeyeScorer(scoring_model,source_vocabs,target_vocab)
        ## call to new string function
        scorer.string_scorer("this is my sentence","this is the second sentence")

def main():
    params = arguments.ConfigArgumentParser(description='Score data with an existing model.')
    arguments.add_score_cli_args(params)
    args = params.parse_args()
    setup_main_logger(file_logging=False, console=True, level=args.loglevel)  # pylint: disable=no-member
    score(args)

if __name__ == "__main__":
    main()

The important part is the new class SockeyeScorer defined at the top with the method string_scorer, which relies on variations of the standard data readers that operate from file input.
I realize this is a bit of a mess, and I am happy to elaborate on details as needed. Those familiar with the score.py script should be able to see what's going on, and I'm wondering if there is something obvious that I should be doing/not doing (when I compare the score_iter instance that my version creates, it looks identical to that in the original scoring script).
In general, it would be great to add functionality to score that allows for scoring from string input/output rather than having to build these rather elaborate data readers from file.
For background, I'm running this in conda on a Mac using Sockey 1.18.85 as per the requirements for ParaBank here: https://github.com/decompositional-semantics-initiative/ParaBank-rewriter/blob/master/requirement.txt .

The easiest way as I see it is to create a special class STDINSequenceReader that extends SequenceReader, which will then split lines from sys.stdin on [TAB], returning (source, target) pairs. You can then instantiate this subclass with a special command-line switch, or say if --source is set to - (a UNIX standard for STDIN).

But wouldn't that still require passing input from STDIN and the CLI?
The class class SimpleSequenceReader(Iterable) that I wrote above seems to be in the same spirit, in that it extends SequenceReader, but instead can take arbitrary strings created on the fly by other python functions (not necessarily from stdin).
I guess what I'm after is being able to use the scorer model more dynamically inside of other python programs that pass around strings from other python functions.

Yes, looking over your code (I hadn't looked closely when I first responded), what you have is in the right spirit.
I'm not sure why it's segfaulting, but I'd try inserting some prints so you can get a lower-level look at what's going on.
Why do you need StringDataInput? Won't the raw batched sampler work?

The StringDataInput is my modified version of the BatchedRawParallelSampleIter (Is this the same sampler that you had in mind above?) from data_io.py. The issue with the latter in my case seems to be here:
https://github.com/awslabs/sockeye/blob/2d458b2a2a4b05b3437a1895209675969bf6befd/sockeye/data_io.py#L1615
where it encodes the src/tgt input using the function create_sequence_readers, which relies on a file rendering of the input and output (in my script I have a new function with the same name just below SimpleSequenceReader that instead does the same thing working with a string).
When I print out the batch data created by all of my new machinery and new sampler, it appears to be identical to that produced by BatchedRawParallelSamplerIter, which is the really puzzling part.
After more investigating, I found that if I override the original scoring function score with the same input in my new SockeyeScorer class, I also get scary errors (which might be correlated with the original problem). For example, keeping everything else constant:
class SockeyeScorer(Scorer):
    """A seq2seq model based on sockeye"""
    def __init__(self,model: ScoringModel,
                 source_vocabs: List[vocab.Vocab],
                 target_vocab: vocab.Vocab) -> None:
        Scorer.__init__(self,model,source_vocabs,target_vocab)

        ## keeps the original vocabulary around to build data in string scorer
        self.orig_source_vocab = source_vocabs
        self.orig_target_vocab = target_vocab

    def string_scorer(self,src,target):
        """Redefined the scoring function

        :param src: the source text
        :param target: the target text
        """
        left_side = src.split()
        right_side = target.split()

        ## custom bucket size
        bucket = (100,100)

        #######################################
        # ON-THE-FLY DATA CREATE FROM STRINGS #
        #######################################

        data_loader = RawParallelDatasetLoader(buckets=[bucket],
                                           eos_id=self.orig_target_vocab[C.EOS_SYMBOL],
                                           pad_id=C.PAD_ID,
                                           skip_blanks=False)

        # special score iterator that builds from strings as opposed to from file

        score_iter = StringDataInput(data_loader=data_loader,
                                           sources=left_side,
                                           target=right_side,
                                           source_vocabs=self.orig_source_vocab,
                                           target_vocab=self.orig_target_vocab,
                                           bucket=bucket,
                                           batch_size=1,
                                           max_lens=(100,100),
                                           #num_factors=0
                                           )

        ## standard run through batch=1
        for batch_no,batch in enumerate(score_iter):

            ## attempt at scoring
            scores = self.model.run(batch)[0] ## raises seg fault and occasionally other errors

    def score(self,
              score_iter,
              output_handler: OutputHandler):

        total_time = 0.
        sentence_no = 0
        batch_no = 0
        for batch_no, batch in enumerate(score_iter, 1):
            batch_tic = time.time()

            # Run the model and get the outputs
            scores = self.model.run(batch)[0]
            return

Called in the following way:
def score(args: argparse.Namespace):

    setup_main_logger(file_logging=False, console=not args.quiet)

    utils.log_basic_info(args)

    with ExitStack() as exit_stack:

        ####################################
        # SAME AS IN ORIGINAL SCORE SCRIPT #
        ####################################

        context = utils.determine_context(device_ids=args.device_ids,
                                          use_cpu=args.use_cpu,
                                          disable_device_locking=args.disable_device_locking,
                                          lock_dir=args.lock_dir,
                                          exit_stack=exit_stack)
        if args.batch_type == C.BATCH_TYPE_SENTENCE:
            check_condition(args.batch_size % len(context) == 0, "When using multiple devices the batch size must be "
                                                                 "divisible by the number of devices. Choose a batch "
                                                                 "size that is a multiple of %d." % len(context))
        logger.info("Scoring Device(s): %s", ", ".join(str(c) for c in context))

        args.no_bucketing = True
        args.bucket_width = 10

        score_iter, source_vocabs, target_vocab, model_config = get_data_iters_and_vocabs(
            args=args,
            model_folder=args.model)

        scoring_model = ScoringModel(config=model_config,
                                             model_dir=args.model,
                                             context=context,
                                             provide_data=score_iter.provide_data,
                                             provide_label=score_iter.provide_label,
                                             default_bucket_key=score_iter.default_bucket_key,
                                             score_type=args.score_type,
                                             length_penalty=inference.LengthPenalty(alpha=args.length_penalty_alpha,
                                                                                beta=args.length_penalty_beta),
                                             softmax_temperature=args.softmax_temperature)

        ######################
        # NEW SCORING MODEL  #
        ######################

        ## call to new sockeye scorer model defined above
        scorer = SockeyeScorer(scoring_model,source_vocabs,target_vocab)
        ## call to new string function
        ### NEW FUNCTION TURNED OFF
        #scorer.string_scorer("this is my sentence","this is the second sentence")

        ## NORMAL SCORING FUNCTION FROM ORIGINAL score.py
        scorer.score(score_iter=score_iter,
                     output_handler=get_output_handler(output_type=args.output_type,
                                                       output_fname=args.output))

I get:
[INFO:sockeye.utils] MXNet version 1.3.1, path /Users/kyler/miniconda2/envs/rewrite_logic/lib/python3.6/site-packages/mxnet/__init__.py
[INFO:sockeye.utils] Command: /Users/kyler/projects/rewrite_logic/rewrite_logic/sockeyeSeq2Seq.py --model /Users/kyler/third_party/ParaBank-rewriter/ --use-cpu --target tgt.txt --source src.txt --batch-size 1
[INFO:sockeye.utils] Arguments: Namespace(batch_size=1, batch_type='word', config=None, device_ids=[-1], disable_device_locking=False, length_penalty_alpha=1.0, length_penalty_beta=0.0, lock_dir='/tmp', loglevel='INFO', max_seq_len=None, model='/Users/kyler/third_party/ParaBank-rewriter/', num_words=(0, 0), output=None, output_type='score', pad_vocab_to_multiple_of=None, quiet=False, score_type='neglogprob', shared_vocab=False, softmax_temperature=None, source='src.txt', source_factor_vocabs=[], source_factors=[], source_vocab=None, target='tgt.txt', target_vocab=None, use_cpu=True, word_min_count=(1, 1))
[INFO:__main__] Scoring Device(s): cpu(0)
[INFO:sockeye.model] ModelConfig loaded from "/Users/kyler/third_party/ParaBank-rewriter/config"
[INFO:sockeye.vocab] Vocabulary (15989 words) loaded from "/Users/kyler/third_party/ParaBank-rewriter/vocab.src.0.json"
[INFO:sockeye.vocab] Vocabulary (13585 words) loaded from "/Users/kyler/third_party/ParaBank-rewriter/vocab.trg.0.json"
[INFO:sockeye.data_io] ==============================
[INFO:sockeye.data_io] Creating scoring data iterator
[INFO:sockeye.data_io] ==============================
[INFO:sockeye.model] Config[_frozen=True, config_data=Config[_frozen=True, data_statistics=Config[_frozen=True, average_len_target_per_bucket=[6.536779555828388, 12.695111308360453, 21.113829676281792, 29.702584587020255, 37.97110374449988, 46.07846054633612, 53.85288055852154, 61.082317040412825, 68.07096396771524, 74.83928896899513, 78.72540254824904, 81.3454750934538], buckets=[(9, 10), (18, 20), (27, 30), (36, 40), (45, 50), (54, 60), (63, 70), (72, 80), (81, 90), (90, 100), (99, 100), (100, 100)], length_ratio_mean=1.0645400645552539, length_ratio_std=0.29834278125977903, max_observed_len_source=100, max_observed_len_target=100, num_discarded=408226, num_sents=56657132, num_sents_per_bucket=[23211115, 18449333, 6644559, 3384680, 1940286, 1186265, 741384, 465335, 302562, 197516, 122594, 11503], num_tokens_source=881889667, num_tokens_target=869418202, num_unks_source=0, num_unks_target=0, size_vocab_source=15989, size_vocab_target=13585], max_seq_len_source=100, max_seq_len_target=100, num_source_factors=1, source_with_eos=True], config_decoder=Config[_frozen=True, attention_config=Config[_frozen=True, config_coverage=None, dtype=float32, input_previous_word=False, is_scaled=False, layer_normalization=False, num_heads=None, num_hidden=1024, query_num_hidden=1024, source_num_hidden=1024, type=dot], attention_in_upper_layers=False, context_gating=False, dtype=float32, enc_last_hidden_concat_to_embedding=False, hidden_dropout=0.2, layer_normalization=False, max_seq_len_source=100, rnn_config=Config[_frozen=True, cell_type=lstm, dropout_inputs=0.0, dropout_recurrent=0.0, dropout_states=0.0, dtype=float32, first_residual_layer=2, forget_bias=0.0, lhuc=False, num_hidden=1024, num_layers=6, residual=False], state_init=last, state_init_lhuc=False], config_embed_source=Config[_frozen=True, dropout=0.0, dtype=float32, factor_configs=None, num_embed=512, num_factors=1, source_factors_combine=concat, vocab_size=15989], config_embed_target=Config[_frozen=True, dropout=0.0, dtype=float32, factor_configs=None, num_embed=512, num_factors=1, source_factors_combine=concat, vocab_size=13585], config_encoder=Config[_frozen=True, conv_config=None, dtype=float32, reverse_input=False, rnn_config=Config[_frozen=True, cell_type=lstm, dropout_inputs=0.0, dropout_recurrent=0.0, dropout_states=0.0, dtype=float32, first_residual_layer=2, forget_bias=0.0, lhuc=False, num_hidden=1024, num_layers=6, residual=False]], config_loss=Config[_frozen=True, label_smoothing=0.1, name=cross-entropy, normalization_type=valid, vocab_size=13585], lhuc=False, vocab_source_size=15989, vocab_target_size=13585, weight_normalization=False, weight_tying=False, weight_tying_type=None]
[INFO:sockeye.encoder] sockeye.encoder.EncoderSequence dtype: float32
[INFO:sockeye.encoder] sockeye.encoder.ConvertLayout dtype: float32
[INFO:sockeye.encoder] sockeye.encoder.BiDirectionalRNNEncoder dtype: float32
[INFO:sockeye.encoder] sockeye.encoder.RecurrentEncoder dtype: float32
[INFO:sockeye.encoder] sockeye.encoder.RecurrentEncoder dtype: float32
[INFO:sockeye.encoder] sockeye.encoder.RecurrentEncoder dtype: float32
[INFO:sockeye.encoder] sockeye.encoder.ConvertLayout dtype: float32
[INFO:sockeye.decoder] sockeye.decoder.RecurrentDecoder dtype: float32
[INFO:sockeye.encoder] sockeye.encoder.Embedding dtype: float32
[INFO:sockeye.encoder] sockeye.encoder.Embedding dtype: float32
[INFO:sockeye.model] Loaded params from "/Users/kyler/third_party/ParaBank-rewriter/params.best"
[INFO:sockeye.data_io] Created bucketed parallel data set. Introduced padding: source=95.0% target=94.0%)
python(21585,0x700007b32000) malloc: *** error for object 0x7fe243fe9e80: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6

Shouldn't
scorer.string_scorer("this is my sentence", "this is the second sentence")

be
scorer.string_scorer(["this is my sentence"], "this is the second sentence")

Sockeye expects a list of source strings, which is how it handles source factors. When factors are not used, it is a list of length one. The target side is always just a plain string since there are not target side factors.
I'm not sure what else to suggest besides this, but it seems some ground-up debugging should be able to resolve what's going on.

Yeh, I think I handled this using the following in the code above (where the source is in a list; I'm generally aware of this point about source factors):
source_sequence_readers = [SimpleSequenceReader(sources,vocab_sources[0],add_eos=True)] ## SEE ABOVE
target_sequence_reader = SimpleSequenceReader(target,vocab_target,add_bos=True)

I'll have another look. But yes, whatever it is, it is a fairly annoying bug indeed! Thanks for having a look.

An easier approach would be to avoid the iterator functionality altogether, since you don't need it. Just construct the batch manually yourself. See the score() function in scoring.py, which is where the iterator is used. You would just extend string_scorer() function to build it manually.

Closing as scoring has been rewritten in Sockeye 2 and issue has not been updated for > 1 year.

authors: fhieber mjpost yakazimir

repo: awslabs/sockeye


text:
Is this compatible with async SQLAlchemy ORM?

Hello,
Would this package work with the (somewhat) new version of the SQLAlchemy ORM which is async?
They use different engine classes (AsyncEngine + AsyncSession VS Engine + Session)
Thanks for proj.

L

Hi @luisroel91, I am not 100% sure, but I think it should work. However, the routes themselves for the SQLAlchemyRouter are not async functions which could be problematic. However, this can be fixed.

source: gharchive/issue

repo: awtkns/fastapi-crudrouter

license: MIT

license_type: permissive

license_source: github-api


text:
The release workflow is blocked in a fork repo

Propose-a-new-feature
Description

https://github.com/Flouse/axon/actions/runs/5965844552
Describe the solution you'd like
A forked repo may want to release its own version for testing.

[x] fix: a forked repo doesn't have self-hosted runners
-> Maybe add a condition to handle this

fixed by
https://github.com/axonweb3/axon/blob/9ce5aa0cc8e554f3a655dc9dc6394cdad69b96ff/.github/workflows/build_image_ghcr.yml#L13-L19


text:
Smarter handling of error/validation for xquery files needed

When editing xquery files there is a noticeable lag because of the requests being sent to the server

If you do automatic validation on them, you should try doing that on a thread or posting events on a java.util.Timer.

Added extra thread

authors: mgaerber raducoravu

repo: axxepta/project-argon

license: mit

license_source: bigquery


text:
The Camera view incorrect when designResolutionSize changed

axys version:
devices test on:
developing environments

NDK version: r19c
Xcode version: 12.4
VS/msvctools version: Visual Studio 2019(16.11)/14.29.30133
cmake version:
Steps to Reproduce:

RT, this issue introduced since #653 merged.
@DelinWorks Could you take a look?
Projection is 3D

could you elaborate more on what you mean by designResolutionSize changed?

Seems there have no chance to update fov when winSize changed: https://github.com/axys1/axys/blob/dev/core/2d/CCCamera.cpp#L457

Before increase winSize.height:

After increase winSize.height

The reproduce example code could be:

set init frame and design size

glview->setFrameSize(1280, 720);
glview->setDesignResolutionSize(1280, 720, ResolutionPolicy::NO_BORDER);

run a scene with some nodes, i.e. label or sprite
increase design resolution and frame size at next frame

glview->setFrameSize(1920, 1080);
glview->setDesignResolutionSize(1920, 1080, ResolutionPolicy::NO_BORDER);

Yeah I was able to replicate that bug, I'll fix it no worries


text:
Click 'x usages' in vscode raises an error

Already fixed

repo: aya-prover/aya-dev


text:
Add-ons compatibility

On the Android version of Firefox, i can not change anything (proxy etc) other than the default configuration of "Bandwidth Hero", because i can not access the add-ons settings page. Other extensions like uBlock Origin works well and their add-ons settings are opened on a new page

Thank you for reporting it!
I'll try to take a look on it.

authors: AndhikaWB ayastreb

repo: ayastreb/bandwidth-hero


text:
Send initial zero value

Hey Alexander,
similar to issue #8 I'd like to propose to send a zero value for metrics that haven't been processed before. The reason for this is that I'd like to create a dashboard based on metrics that ideally never have a value greater zero (e.g. a counter for a "this shouldn't happen" exception) but I still want to save some money and not send a zero on every interval.
Let me know what you think and how I might help. Thanks.
Dimitri

Thanks for that. Feel free to issue a PR similar to https://github.com/azagniotov/codahale-aggregated-metrics-cloudwatch-reporter/pull/9

authors: azagniotov zersetz-end


text:
Problem with Websocket (Django)

Hey community!
When i post data to MessageCreateAPIView class, the data is being saved in database but websocket is not working properly, after sending the data following error appears:
ConnectionError at /chat/rooms/message/create/
Connection closed by server.
Do you have any ideas to fix this?

Already fixed, the problem was that new version of channel requires daphne. You need to install it and add to apps in settings

repo: azamatovuser/Telegram-Clone-API


text:
L10n fr

Hey again,
As mentioned in #56, this is the second step of our recent work on globalizing and localizing expressworks. This includes the commits referenced in #56, which will "disappear" when/if you merge that one first, to leave only FR-related commits for actual localization.
I remain, naturally, available for any discussion on this.
Looking forward to our getting that in,
Best,

And yay, this auto-merges again!

@tdd I merged both PRs and published at 2.0.0... thank you! can you double check it again? or during your nodeschool event?

Hey Azat, thanks a ton! npm does show the package.
I realize perhaps it's time to update that screenshot from the README :wink:

haha

source: gharchive/pull-request

authors: azat-co tdd

repo: azat-co/expressworks


text:
fix: handle more complex expressions in sort-objects

Description

3e987ae7e6fa9fc21caa7e23fb8b17d9ece027f0 fixes references to dependencies/earlier params (#9), but not in some more complex cases. Examples are provided in tests.
Additional context

Do you think this is a good approach? I think we should try to search all of the nested nodes for Identifiers, which would be pushed to the dependencies array. I'm not sure I covered all the possible cases with this approach.
The tests also probably need some more work on the semantics :)

What is the purpose of this pull request?

[x] Bug fix
[ ] New Feature
[ ] Documentation update
[ ] Other

Before submitting the PR, please make sure you do the following

[x] Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
[x] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).
[x] Ideally, include relevant tests that fail without this PR but pass with it.
[x] Read contribution guidelines.

@azat-io Wow, that was fast :) I'll try to follow up with a PR adding the missing test cases and possibly doing something about making this auto fixable.

repo: azat-io/eslint-plugin-perfectionist


text:
Change in workflow

Users of the previous AZGS toolbar made features for ContactsAndFaults first, then used an ArcObjects tool call Construct Polygons to make features for MapUnits out of the ContactsAndFaults feature.
This isn't what we've so far designed and requires a rethink.
Questions:

Can we create polys from polylines in the Pro SDK?
How to implement the linear workflow relationship between ContactsAndFaults and MapUnits?

The answer to (1) appears to be "yes". An example can be found here: https://github.com/Esri/arcgis-pro-sdk-community-samples/blob/master/Geometry/Constructing-Geometries/ConstructingGeometries_CSharp/createPolygons.cs
Question (2) requires some discussion. I wasn't aware of this linear relationship and have been running on the assumption that we'd provide create/edit tools for both feature classes in the ribbon, and the user could decide what they wanted to work on. But if there is an expected linear relationship to this, we need to design the ribbon tools, and possibly access to them, around that. More questions come up:

Should a user be able to directly edit Map Unit geometries or should they only be created from C&F geometries?
What is the symbology relationship between MU and C&F features?

Right now, I'm thinking that both tools will be available in the ribbon all the time. Clicking the MU tool will no longer allow you to sketch on the map. Instead, it will allow you to select C&F features to plug into the polygon builder. If there are no C&F features, there will simply be nothing to select with the MU tool. There will be no relationship between the symbologies of C&F and MU features.

The specifics of this issue have been addressed in #20 and #22.
Should we close this one as redundant?

repo: azgs/geomapmaker


text:
azkaban.executor.ExecutorManagerException: java.io.IOException: executor became inactive before setting up the flow 8

This error is reported for all tasks performed, including the official example. The database executor is active. What is the reason for this。
The details of the error are as follows：
2019/03/09 10:02:42.736 +0800 INFO [ExecutorManager] [Azkaban] Using dispatcher for execution id :8
2019/03/09 10:02:42.943 +0800 ERROR [ExecutorManager] [Azkaban] Rolling back executor assignment for execution id:8
azkaban.executor.ExecutorManagerException: java.io.IOException: executor became inactive before setting up the flow 8
at azkaban.executor.ExecutorApiGateway.callWithExecutionId(ExecutorApiGateway.java:80)
at azkaban.executor.ExecutorApiGateway.callWithExecutable(ExecutorApiGateway.java:43)
at azkaban.executor.ExecutorManager.dispatch(ExecutorManager.java:1113)
at azkaban.executor.ExecutorManager.access$600(ExecutorManager.java:73)
at azkaban.executor.ExecutorManager$QueueProcessorThread.selectExecutorAndDispatchFlow(ExecutorManager.java:1344)
at azkaban.executor.ExecutorManager$QueueProcessorThread.processQueuedFlows(ExecutorManager.java:1316)
at azkaban.executor.ExecutorManager$QueueProcessorThread.run(ExecutorManager.java:1254)
Caused by: java.io.IOException: executor became inactive before setting up the flow 8
at azkaban.executor.ExecutorApiGateway.callForJsonObjectMap(ExecutorApiGateway.java:108)
at azkaban.executor.ExecutorApiGateway.callWithExecutionId(ExecutorApiGateway.java:78)
... 6 more
2019/03/09 10:02:42.951 +0800 WARN [ExecutorManager] [Azkaban] Executor node154:42491 (id: 3), active=true responded with exception for exec: 8
azkaban.executor.ExecutorManagerException: azkaban.executor.ExecutorManagerException: java.io.IOException: executor became inactive before setting up the flow 8
at azkaban.executor.ExecutorManager.dispatch(ExecutorManager.java:1119)
at azkaban.executor.ExecutorManager.access$600(ExecutorManager.java:73)
at azkaban.executor.ExecutorManager$QueueProcessorThread.selectExecutorAndDispatchFlow(ExecutorManager.java:1344)
at azkaban.executor.ExecutorManager$QueueProcessorThread.processQueuedFlows(ExecutorManager.java:1316)
at azkaban.executor.ExecutorManager$QueueProcessorThread.run(ExecutorManager.java:1254)
Caused by: azkaban.executor.ExecutorManagerException: java.io.IOException: executor became inactive before setting up the flow 8
at azkaban.executor.ExecutorApiGateway.callWithExecutionId(ExecutorApiGateway.java:80)
at azkaban.executor.ExecutorApiGateway.callWithExecutable(ExecutorApiGateway.java:43)
at azkaban.executor.ExecutorManager.dispatch(ExecutorManager.java:1113)
... 4 more
Caused by: java.io.IOException: executor became inactive before setting up the flow 8
at azkaban.executor.ExecutorApiGateway.callForJsonObjectMap(ExecutorApiGateway.java:108)
at azkaban.executor.ExecutorApiGateway.callWithExecutionId(ExecutorApiGateway.java:78)
... 6 more

maybe executor  is down

Take a look at the startup log to see if the executor is really alive, and judging from the error messages, the connection did time out when web connection exec was reconfirmed after the selection

Take a look at the startup log to see if the executor is really alive, and judging from the error messages, the connection did time out when web connection exec was reconfirmed after the selection

maybe executor is down

The problem was solved temporarily, but not in a reasonable way:
I had no intention of trying to process the request: http://node151:12321/executor? Activate action=activate this address so that all my tasks can run as normal.
I tried again and again, but the error was not resolved when the execution port was not fixed.
Is this because something is not configured properly?

Is the log level debug or info?

Is the log level debug or info?

INFO

property active in class FlowRunnerManager is false default, there should be true by some way, it confuse me ,too

We had this problem. We run in ECS with one container called azkaban-webserver and another called azkaban-executor. In our db there is only one executor named "azkaban-executor". We suspect during upgrade the old azkaban-executor must have marked the executor inactive in database.
As a workaround we changed our container entrypoint to start azkaban-executor and then call curl http://localhost:port/executor?action=activate to make sure the new executor is active.

你好! 解决了吗?

Take a look at the startup log to see if the executor is really alive, and judging from the error messages, the connection did time out when web connection exec was reconfirmed after the selection

maybe executor is down

The problem was solved temporarily, but not in a reasonable way:
I had no intention of trying to process the request: http://node151:12321/executor? Activate action=activate this address so that all my tasks can run as normal.
I tried again and again, but the error was not resolved when the execution port was not fixed.
Is this because something is not configured properly?

Take a look at the startup log to see if the executor is really alive, and judging from the error messages, the connection did time out when web connection exec was reconfirmed after the selection

maybe executor is down

The problem was solved temporarily, but not in a reasonable way:
I had no intention of trying to process the request: http://node151:12321/executor? Activate action=activate this address so that all my tasks can run as normal.
I tried again and again, but the error was not resolved when the execution port was not fixed.
Is this because something is not configured properly?

你好! 找着原因了吗? 怎么解决的呢, 方便说下吗?

do request by this
curl http://${executorHost}:${executorPort}/executor?action=activate

I also got this error, because I don't active executor node by call curl http://${executorHost}:${executorPort}/executor?action=activate.
I did this, add executor node from the table executors, this make me got the error.
call curl http://${executorHost}:${executorPort}/executor?action=activate is work.

I also got the error but the issue was in DB where the executors config lies. It has canonical hostname instead of The IP address and web node was not able to communicate with the executor node.
there is a configuration in executor azkaban.server.hostname If it is not set the executor register itself with canonical hostname(hotname -f). and for this there should be the host entry mapping in web node  fopr the executor node <hostname -i> . If the above config is set the executor registers itself with the provided value of in config zkaban.server.hostname

I also got the error but the issue was in DB where the executors config lies. It has canonical hostname instead of The IP address and web node was not able to communicate with the executor node.
there is a configuration in executor azkaban.server.hostname If it is not set the executor register itself with canonical hostname(hotname -f). and for this there should be the host entry mapping in web node  fopr the executor node <hostname -i> . If the above config is set the executor registers itself with the provided value of in config zkaban.server.hostname

repo: azkaban/azkaban


text:
Azkaban Multiple executor mode error

Hi,
Recieved following error when configured azkaban to run in multiple executor mode.
azkaban.executor.ExecutorManagerException: azkaban.executor.ExecutorManagerException: java.io.IOException: Error loading flow with exec 19
at azkaban.executor.ExecutorManager.dispatch(ExecutorManager.java:1745)
at azkaban.executor.ExecutorManager.submitExecutableFlow(ExecutorManager.java:1041)
at azkaban.webapp.servlet.ExecutorServlet.ajaxExecuteFlow(ExecutorServlet.java:885)
at azkaban.webapp.servlet.ExecutorServlet.ajaxAttemptExecuteFlow(ExecutorServlet.java:844)
at azkaban.webapp.servlet.ExecutorServlet.handleAJAXAction(ExecutorServlet.java:160)
at azkaban.webapp.servlet.ExecutorServlet.handleGet(ExecutorServlet.java:84)
at azkaban.webapp.servlet.LoginAbstractAzkabanServlet.doGet(LoginAbstractAzkabanServlet.java:121)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: azkaban.executor.ExecutorManagerException: java.io.IOException: Error loading flow with exec 19
at azkaban.executor.ExecutorManager.callExecutorServer(ExecutorManager.java:1071)
at azkaban.executor.ExecutorManager.dispatch(ExecutorManager.java:1739)
... 21 more
Caused by: java.io.IOException: Error loading flow with exec 19
at azkaban.executor.ExecutorManager.callExecutorForJsonObject(ExecutorManager.java:1143)
at azkaban.executor.ExecutorManager.callExecutorServer(ExecutorManager.java:1124)
at azkaban.executor.ExecutorManager.callExecutorServer(ExecutorManager.java:1068)
... 22 more
I am using current build of 3.0.0 build for Azkaban Web Server and Azkaban Executer server
All DB set up has been done for Web server as well as for executers servers as well.
Please suggest?
Thanks,
Gunesh P.

@gunesh4u I met the error too.How to fix it ?

Can you share error message/log?
Thanks,
Gp
On Fri, 17 Nov 2017 at 1:05 PM, Leno Bill notifications@github.com wrote:

@gunesh4u https://github.com/gunesh4u I met the error too.How to fix it
?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/azkaban/azkaban/issues/773#issuecomment-345167688,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AV2-Wyy7RWuthUGTNB94FqZ4ZLEyttxjks5s3TcpgaJpZM4KaNUY
.

--
Sent from Gmail Mobile

@gunesh4u Thanks ！

[wherehows@mesos1 bin]$ 2017-11-17 16:08:59 INFO  LoginAbstractAzkabanServlet:169 - 10.129.3.1 azkaban "GET /executor projectId=3&project=t3&ajax=executeFlow&flow=test_d3_azkaban&disabled=%5B%5D&failureEmailsOverride=false&successEmailsOverride=false&failureAction=finishCurrent&failureEmails=&successEmails=&notifyFailureFirst=false&notifyFailureLast=false&concurrentOption=skip HTTP/1.1" browser
2017-11-17 16:08:59 INFO  ExecutorManager:939 - Submitting execution flow test_d3_azkaban by azkaban
2017-11-17 16:08:59 INFO  ExecutionFlowDao:72 - Flow given test_d3_azkaban given id 9
2017-11-17 16:08:59 INFO  ExecutorServlet:115 - User  has called action execute on 9
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - azkaban.executor.ExecutorManagerException: Error loading flow with exec 9
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at azkaban.execapp.FlowRunnerManager.submitFlow(FlowRunnerManager.java:324)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at azkaban.execapp.ExecutorServlet.handleAjaxExecute(ExecutorServlet.java:276)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at azkaban.execapp.ExecutorServlet.doGet(ExecutorServlet.java:124)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.Server.handle(Server.java:326)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
2017-11-17 16:08:59 ERROR ExecutorServlet:279 - Error loading flow with exec 9
azkaban.executor.ExecutorManagerException: Error loading flow with exec 9
at azkaban.execapp.FlowRunnerManager.submitFlow(FlowRunnerManager.java:324)
at azkaban.execapp.ExecutorServlet.handleAjaxExecute(ExecutorServlet.java:276)
at azkaban.execapp.ExecutorServlet.doGet(ExecutorServlet.java:124)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
2017-11-17 16:08:59 ERROR ExecutorManager:1378 - Rolling back executor assignment for execution id:9
azkaban.executor.ExecutorManagerException: java.io.IOException: Error loading flow with exec 9
at azkaban.executor.ExecutorApiGateway.callWithExecutionId(ExecutorApiGateway.java:78)
at azkaban.executor.ExecutorApiGateway.callWithExecutable(ExecutorApiGateway.java:43)
at azkaban.executor.ExecutorManager.dispatch(ExecutorManager.java:1375)
at azkaban.executor.ExecutorManager.submitExecutableFlow(ExecutorManager.java:1013)
at azkaban.webapp.servlet.ExecutorServlet.ajaxExecuteFlow(ExecutorServlet.java:943)
at azkaban.webapp.servlet.ExecutorServlet.ajaxAttemptExecuteFlow(ExecutorServlet.java:903)
at azkaban.webapp.servlet.ExecutorServlet.handleAJAXAction(ExecutorServlet.java:175)
at azkaban.webapp.servlet.ExecutorServlet.handleGet(ExecutorServlet.java:95)
at azkaban.webapp.servlet.LoginAbstractAzkabanServlet.doGet(LoginAbstractAzkabanServlet.java:123)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:770)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at org.mortbay.jetty.security.SslSocketConnector$SslConnection.run(SslSocketConnector.java:713)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.io.IOException: Error loading flow with exec 9
at azkaban.executor.ExecutorApiGateway.callForJsonObjectMap(ExecutorApiGateway.java:106)
at azkaban.executor.ExecutorApiGateway.callWithExecutionId(ExecutorApiGateway.java:76)
... 24 more
2017-11-17 16:08:59 INFO  FlowRunnerManager:818 - # of executing flows: 0
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - azkaban.executor.ExecutorManagerException: azkaban.executor.ExecutorManagerException: java.io.IOException: Error loading flow with exec 9
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at azkaban.executor.ExecutorManager.dispatch(ExecutorManager.java:1381)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at azkaban.executor.ExecutorManager.submitExecutableFlow(ExecutorManager.java:1013)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at azkaban.webapp.servlet.ExecutorServlet.ajaxExecuteFlow(ExecutorServlet.java:943)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at azkaban.webapp.servlet.ExecutorServlet.ajaxAttemptExecuteFlow(ExecutorServlet.java:903)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at azkaban.webapp.servlet.ExecutorServlet.handleAJAXAction(ExecutorServlet.java:175)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at azkaban.webapp.servlet.ExecutorServlet.handleGet(ExecutorServlet.java:95)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at azkaban.webapp.servlet.LoginAbstractAzkabanServlet.doGet(LoginAbstractAzkabanServlet.java:123)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:770)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.Server.handle(Server.java:326)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.jetty.security.SslSocketConnector$SslConnection.run(SslSocketConnector.java:713)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - Caused by: azkaban.executor.ExecutorManagerException: java.io.IOException: Error loading flow with exec 9
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at azkaban.executor.ExecutorApiGateway.callWithExecutionId(ExecutorApiGateway.java:78)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at azkaban.executor.ExecutorApiGateway.callWithExecutable(ExecutorApiGateway.java:43)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at azkaban.executor.ExecutorManager.dispatch(ExecutorManager.java:1375)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	... 22 more
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - Caused by: java.io.IOException: Error loading flow with exec 9
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at azkaban.executor.ExecutorApiGateway.callForJsonObjectMap(ExecutorApiGateway.java:106)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	at azkaban.executor.ExecutorApiGateway.callWithExecutionId(ExecutorApiGateway.java:76)
2017-11-17 16:08:59 ERROR StdOutErrRedirect:55 - 	... 24 more


text:
株式会社カブクです

会社情報
株式会社カブク
従業員数: 約30名
所在地:
〒160-0022
東京都新宿区新宿5-17-17 渡菱ビル1F
（東証一部上場の双葉電子工業株式会社の連結子会社です）
会社/サービスURL
コーポレートサイト
https://www.kabuku.co.jp/
JSer.infoで何度か取り上げていただいたことがありますが、Developers Blogをご覧いただくと、在籍しているエンジニアの雰囲気が伝わるかと思います。
https://www.kabuku.co.jp/developers
Connect
https://connect.kabuku.io/
特注品などを作りたい顧客企業を、グローバルな工場ネットワークとつなぐ（技術購買のアウトソーシング）サービスです
Rinkak
https://www.rinkak.com/
ユーザーが3Dデータをアップロードし、そこから3Dプリントプロダクトを試作したり、売ったり、買ったりできるサービスです
連絡した理由
以下のような点で、azuさんのご希望にマッチするのではないかと考え、ご連絡させていただきました。

B2B
拡大期の企業

昨年双葉電子工業株式会社の子会社となり、スタートアップの時期を終えて拡大しようとしているところです

課題

ものづくりの民主化

大規模な製造設備を持たない個人、法人でも、簡単にものづくりをできるようにする「ものづくりの民主化」が会社としてのビジョンで、この課題をテクノロジーで解決することに取り組んでいます
上記で紹介したConnectやRinkakはこのためのサービスです

製造業のデジタル化

たとえば、以下のような用途に機械学習を適用しています

3Dデータや2Dデータからの情報抽出モデルの構築
センサーデータの分析モデルの調査・研究・開発

その他アピールポイント
エンジニアは上下関係がなくフラットで働きやすい雰囲気だと思います。
書籍の執筆やカンファレンス、勉強会での発表などのアウトプット活動も各人積極的に取り組んでいます。
会社として「自治あっての自由」というのを掲げていて、自由でありながら決してだらしなくはならないところもいいと思っています。
azuさんの求職記事をみた若手エンジニアが、azuさんに来て欲しいという思いが募ったあまりお席をフライングで用意してしまいました。
azuさんのお名前は3Dプリンタでプリントしたものですが、弊社では福利厚生で3Dプリンタが使い放題です。

興味を持っていただけましたら、ぜひお気軽にご返信ください。

ご連絡ありがとうございます。

JSer.infoで何度か取り上げていただいたことがありますが、

はい、以前から存じ上げています。
今回の求職活動中にも一度調べたことがあるのですが、どうしても株式会社カブクで自分が働いているイメージができなかったという感覚があります。
自分があまり物理的なものに興味をもつことが少ないので、製造業と少し距離があるからかもしれません。
申し訳ありませんがクローズさせていただきます。

authors: azu kimamula

repo: azu/open-job-letter


text:
Update deprecation in Build.ts

build.ts has a soon to be deprecated Deno.run  This should be upgraded to Deno.command

@nhrones the deno bundle CLI command is also deprecated and scheduled to be removed with Deno 2.0.
I'll look into getting rid of both.

authors: boywithkeyboard nhrones

repo: azurystudio/cheetah


text:
display functions with processing-sublime

hi benedikt,
first, thank you for this nifty plugin!
did you try to get https://github.com/BoundInCode/Display-Functions working with your plugin? Or do you see any other way to achieve this autocomplete functions?
thx,sebl

pls add autocomplete :)

authors: jhonoryza sebllll

repo: b-g/processing-sublime


text:
client - adding ability to include query string in request

It’s ugly, but not against the specification.  In an indirect, kind of heavily interpreted way, it’s explicitly allowed, see:
http://xmlrpc.scripting.com/spec.html

The format of the URI in the first line of the header is not specified.

This, as far as I can tell, indicates that it CAN include query strings.

I didn't realize I was using outdated files here.  I've got a better one coming up-- sorry about that.

repo: baalexander/node-xmlrpc


text:
adds indented mode to Jsx via parser option

This PR comes from a conversation in facebook/jsx#60 where I proposed adding the ability to use Jade like compact indented JSX syntax via option.
Reasoning.
A lot of Jsx lines are useless end tags. But screen and bandwidth space is precious. And as projects/sources grow ability to focus and spot bugs is often lost in noise especially after inline styles kick in.
Source changes:
No control flow / state changes.
all code is enclosed and activated only when indented:true,locations:true passed in parser ptions
Because Jsx parser is simple all that was needed was adding children based on indentation(for now it uses line and column no from (locations:true) in jsxParseElementAt and disable complaining about missing end tags in jsxReadToken and
All tests pass with and without option.
I also implemented first indent test in Experimental/Jsx/indented dir along with sample
var App  =  <div>
                <drawer>
                    <menu>
                        <item>
                <area>
                    <toolbar>
                    <content>

This will probably later result to faster simpler parsing due to half of tags  and smaller ast trees in web
assembly later but most importantly Jsx UI structure will now be much more easy to understand and bugs easier to spot especially in complex projects.

This would definitely need to be part of the official spec before we're likely to accept it.

I definitely understand. I reimplemented it as JadeX plugin as proposed by facebook guys. Thx

authors: loganfsmyth neurall

repo: babel/babylon


text:
Implement wrapper function for babelfish inconsistency checks

Description
This commit implements a wrapper function for all the checks to detect inconsistency between babelfish catalogs and pg catalogs. As of now, we are only checking the metadata consistency between these catalogs.
Signed-off-by: Roshan Kanwar rskanwar@amazon.com
Issues Resolved
Task: BABEL-4139
Test Scenarios Covered

Use case based - Y

Boundary conditions -

Arbitrary inputs -

Negative test cases -

Minor version upgrade tests -

Major version upgrade tests -

Performance tests -

Tooling impact -

Client tests -

Check List

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is under the terms of the Apache 2.0 and PostgreSQL licenses, and grant any person obtaining a copy of the contribution permission to relicense all or a portion of my contribution to the PostgreSQL License solely to contribute all or a portion of my contribution to the PostgreSQL open source project.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Pull Request Test Coverage Report for Build 8596864378
Details

0 of 0   changed or added relevant lines in 0 files are covered.
38 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-0.05%) to 72.513%

Files with Coverage Reduction
New Missed Lines
%

contrib/babelfishpg_tds/src/backend/tds/tdsutils.c
1
73.39%

contrib/babelfishpg_tsql/src/dbcmds.c
17
71.13%

contrib/babelfishpg_tsql/src/rolecmds.c
20
79.28%

Totals

Change from base Build 8595421107:
-0.05%

Covered Lines:
41101

Relevant Lines:
56681

💛  - Coveralls


text:
[Bug] DTO import重命名逻辑似乎没有效果

正常情况引入是可以编译成功的

但是添加重命名后就会编译错误

而且还有一个问题是如果将引入的类重命名为另一个已引入的类，那么会直接采用被重命名的类

这是三个超接口的定义
@MappedSuperclass
public interface AdditionDeletedInfo {
    /**
     * 删除标志（0代表存在 2代表删除）
     */
    @LogicalDeleted(value = "2", restoredValue = "0")
    int delFlag();
}

@MappedSuperclass
public interface AdditionOperatorInfo {
    /**
     * 创建者
     */
    @Nullable
    @ManyToOne
    @JoinColumn(name = "create_by",foreignKeyType = ForeignKeyType.FAKE)
    SysUser createBy();

    /**
     * 更新者
     */
    @Nullable
    @ManyToOne
    @JoinColumn(name = "update_by", foreignKeyType = ForeignKeyType.FAKE)
    SysUser updateBy();
}

@MappedSuperclass
public interface AdditionTimeInfo {
    /**
     * 创建时间
     */
    @Nullable
    @Column(name = "create_time")
    LocalDateTime createTime();

    /**
     * 更新时间
     */
    @Nullable
    @Column(name = "update_time")
    LocalDateTime updateTime();
}

生成的 TestView 部分代码如下，可以看到最终是直接使用了 AdditionDeletedInfo 和 AdditionTimeInfo，而不是重命名的 AdditionTimeInfo 和 AdditionOperatorInfo
@GeneratedBy(
        file = "src/main/dto/com/example/auth/entity/SysUser.dto"
)
@Data
public class TestView implements View<SysUser> {
    public static final ViewMetadata<SysUser, TestView> METADATA =
        new ViewMetadata<SysUser, TestView>(
            SysUserFetcher.$
               // .other_props()
                .delFlag()
                .createTime()
                .updateTime(),
            TestView::new
    );

   // omit other props and methods

    private int delFlag;

    @Nullable
    private LocalDateTime createTime;

    @Nullable
    private LocalDateTime updateTime;
}

最后第二步重命名 AdditionDeletedInfo 为 ADI 的完整的堆栈信息如下：
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.11.0:compile (default-compile) on project auth: Fatal error compiling: org.babyfish.jimmer.apt.DtoException: Failed to parse "C:\Users\Admin\SoftWare\Idea\IdeaProjects\project\auth\src\main\dto\com\example\auth\entity\SysUser.dto": Error at line 9 of "src/main/dto/com/example/auth/entity/SysUser.dto": Illegal type name "ADI", it is not super type of "com.example.auth.entity.SysUser" -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.11.0:compile (default-compile) on project auth: Fatal error compiling
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
    at org.codehaus.classworlds.Launcher.main (Launcher.java:47)
Caused by: org.apache.maven.plugin.MojoExecutionException: Fatal error compiling
    at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute (AbstractCompilerMojo.java:1143)
    at org.apache.maven.plugin.compiler.CompilerMojo.execute (CompilerMojo.java:193)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
    at org.codehaus.classworlds.Launcher.main (Launcher.java:47)
Caused by: org.codehaus.plexus.compiler.CompilerException: org.babyfish.jimmer.apt.DtoException: Failed to parse "C:\Users\Admin\SoftWare\Idea\IdeaProjects\project\auth\src\main\dto\com\example\auth\entity\SysUser.dto": Error at line 9 of "src/main/dto/com/example/auth/entity/SysUser.dto": Illegal type name "ADI", it is not super type of "com.example.auth.entity.SysUser"
    at org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess (JavaxToolsCompiler.java:198)
    at org.codehaus.plexus.compiler.javac.JavacCompiler.performCompile (JavacCompiler.java:183)
    at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute (AbstractCompilerMojo.java:1140)
    at org.apache.maven.plugin.compiler.CompilerMojo.execute (CompilerMojo.java:193)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
    at org.codehaus.classworlds.Launcher.main (Launcher.java:47)
Caused by: java.lang.RuntimeException: org.babyfish.jimmer.apt.DtoException: Failed to parse "C:\Users\Admin\SoftWare\Idea\IdeaProjects\project\auth\src\main\dto\com\example\auth\entity\SysUser.dto": Error at line 9 of "src/main/dto/com/example/auth/entity/SysUser.dto": Illegal type name "ADI", it is not super type of "com.example.auth.entity.SysUser"
    at com.sun.tools.javac.api.JavacTaskImpl.invocationHelper (JavacTaskImpl.java:168)
    at com.sun.tools.javac.api.JavacTaskImpl.doCall (JavacTaskImpl.java:100)
    at com.sun.tools.javac.api.JavacTaskImpl.call (JavacTaskImpl.java:94)
    at org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess (JavaxToolsCompiler.java:136)
    at org.codehaus.plexus.compiler.javac.JavacCompiler.performCompile (JavacCompiler.java:183)
    at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute (AbstractCompilerMojo.java:1140)
    at org.apache.maven.plugin.compiler.CompilerMojo.execute (CompilerMojo.java:193)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
    at org.codehaus.classworlds.Launcher.main (Launcher.java:47)
Caused by: org.babyfish.jimmer.apt.DtoException: Failed to parse "C:\Users\Admin\SoftWare\Idea\IdeaProjects\project\auth\src\main\dto\com\example\auth\entity\SysUser.dto": Error at line 9 of "src/main/dto/com/example/auth/entity/SysUser.dto": Illegal type name "ADI", it is not super type of "com.example.auth.entity.SysUser"
    at org.babyfish.jimmer.apt.ImmutableProcessor.parseDtoTypes (ImmutableProcessor.java:197)
    at org.babyfish.jimmer.apt.ImmutableProcessor.process (ImmutableProcessor.java:112)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment.callProcessor (JavacProcessingEnvironment.java:1021)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment.discoverAndRunProcs (JavacProcessingEnvironment.java:937)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.run (JavacProcessingEnvironment.java:1265)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing (JavacProcessingEnvironment.java:1380)
    at com.sun.tools.javac.main.JavaCompiler.processAnnotations (JavaCompiler.java:1271)
    at com.sun.tools.javac.main.JavaCompiler.compile (JavaCompiler.java:948)
    at com.sun.tools.javac.api.JavacTaskImpl.lambda$doCall$0 (JavacTaskImpl.java:104)
    at com.sun.tools.javac.api.JavacTaskImpl.invocationHelper (JavacTaskImpl.java:152)
    at com.sun.tools.javac.api.JavacTaskImpl.doCall (JavacTaskImpl.java:100)
    at com.sun.tools.javac.api.JavacTaskImpl.call (JavacTaskImpl.java:94)
    at org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess (JavaxToolsCompiler.java:136)
    at org.codehaus.plexus.compiler.javac.JavacCompiler.performCompile (JavacCompiler.java:183)
    at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute (AbstractCompilerMojo.java:1140)
    at org.apache.maven.plugin.compiler.CompilerMojo.execute (CompilerMojo.java:193)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
    at org.codehaus.classworlds.Launcher.main (Launcher.java:47)
Caused by: org.babyfish.jimmer.dto.compiler.DtoAstException: Error at line 9 of "src/main/dto/com/example/auth/entity/SysUser.dto": Illegal type name "ADI", it is not super type of "com.example.auth.entity.SysUser"
    at org.babyfish.jimmer.dto.compiler.DtoCompiler.exception (DtoCompiler.java:89)
    at org.babyfish.jimmer.dto.compiler.CompilerContext.exception (CompilerContext.java:162)
    at org.babyfish.jimmer.dto.compiler.DtoTypeBuilder.handleAllScalars (DtoTypeBuilder.java:168)
    at org.babyfish.jimmer.dto.compiler.DtoTypeBuilder.<init> (DtoTypeBuilder.java:86)
    at org.babyfish.jimmer.dto.compiler.CompilerContext.add (CompilerContext.java:91)
    at org.babyfish.jimmer.dto.compiler.DtoCompiler.parse (DtoCompiler.java:71)
    at org.babyfish.jimmer.dto.compiler.DtoCompiler.compile (DtoCompiler.java:50)
    at org.babyfish.jimmer.apt.ImmutableProcessor.parseDtoTypes (ImmutableProcessor.java:191)
    at org.babyfish.jimmer.apt.ImmutableProcessor.process (ImmutableProcessor.java:112)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment.callProcessor (JavacProcessingEnvironment.java:1021)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment.discoverAndRunProcs (JavacProcessingEnvironment.java:937)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.run (JavacProcessingEnvironment.java:1265)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing (JavacProcessingEnvironment.java:1380)
    at com.sun.tools.javac.main.JavaCompiler.processAnnotations (JavaCompiler.java:1271)
    at com.sun.tools.javac.main.JavaCompiler.compile (JavaCompiler.java:948)
    at com.sun.tools.javac.api.JavacTaskImpl.lambda$doCall$0 (JavacTaskImpl.java:104)
    at com.sun.tools.javac.api.JavacTaskImpl.invocationHelper (JavacTaskImpl.java:152)
    at com.sun.tools.javac.api.JavacTaskImpl.doCall (JavacTaskImpl.java:100)
    at com.sun.tools.javac.api.JavacTaskImpl.call (JavacTaskImpl.java:94)
    at org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess (JavaxToolsCompiler.java:136)
    at org.codehaus.plexus.compiler.javac.JavacCompiler.performCompile (JavacCompiler.java:183)
    at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute (AbstractCompilerMojo.java:1140)
    at org.apache.maven.plugin.compiler.CompilerMojo.execute (CompilerMojo.java:193)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
    at org.codehaus.classworlds.Launcher.main (Launcher.java:47)
[ERROR]
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :auth

环境：Java21, SpringBoot 3, Jimmer 0.8.24

Try 0.8.31

authors: EmyiaEllen babyfish-ct

repo: babyfish-ct/jimmer


text:
🛑 SM Service is down

In 5bf600d, SM Service (http://smservice.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: SM Service is back up in a0e3a61 after 17 minutes.

repo: bachmannschumacher/upptime


text:
🛑 SM Service is down

In 1a7c9a8, SM Service (http://smservice.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: SM Service is back up in fd88ddc after 22 minutes.


text:
🛑 SM Service is down

In ee05710, SM Service (http://smservice.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: SM Service is back up in b84427c.


text:
🛑 SM Service is down

In 8bd8336, SM Service (http://smservice.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: SM Service is back up in 9d129b4.


text:
I cannot get the similar results with provided resnet-50 (w/o edge) as repoted in paper

I cannot get the similar results with provided resnet-50 (w/o edge run-0) as repoted in paper.

We evaluated all the results with the code provided here.
I re-evaluated the results of ResNet-50 (w/o edge) and got identical scores.
Detailed scores are here.
eval_results.txt

repo: backseason/PoolNet


text:
Google GCS Discovery

Feature Suggestion

Would be great if the Google GCS configuration location had a "discovery" option like the AWS or Azure buckets.
I want to be able to just place configuration items in a bucket so that Backstage can just periodically look for all the matching yaml configuration and load it as necessary.
Possible Implementation

Potentially just copying the way the AWS bucket discovery plugin works:
https://github.com/backstage/backstage/blob/master/plugins/catalog-backend-module-aws/src/processors/AwsS3DiscoveryProcessor.ts
I can't imagine the logic will need to differ too much between them.
Context

For the moment I'm having to create all my configuration ahead of time, so it's like an "eventually consistent" service.
This unfortunately has some minor draw backs and complexities I'm trying to solve.

Anything thoughts on this?

The gcs reader supports search patterns. I think you'll be able to just register a regular location (or having it in your config), that points to https://storage.cloud.google.com/my-bucket/some-folder/* or so, if the catalog files are under that keyspace.
I didn't verify this just now - am not at the computer :)

What name format do the files need to be? All must be catalog.yaml or something along those lines?
I assume this doesn't just scan the bucket recursively either?

@lucasteligioridis I think that generally all of the object storage readers assume that all matching objects are catalog entities, so there's no filtering on filenames, it's all assumed to be entity definitions.
Don't know whether it's recursive or not tbh, but I think we try to keep it close to the other providers and what the gcs library does by default.
Overall I'd say It'd be nice to add GCS discovery. Probably just focus on the provider though, as in the equivalent of https://github.com/backstage/backstage/blob/master/plugins/catalog-backend-module-aws/src/providers/AwsS3EntityProvider.ts. With a provider you can pass more granular options compared to the processor approach.

Thanks @Rugvip will test this out. For my plan to work correctly, I'd at least need recursive searching to work so we can create our catalog files asynchronously in the bucket with specific namespaces/directories to prevent clobbering of files.
I DO believe I tried this and it didn't work recursively, which meant I had to pre-populate my configuration in backstage to list all the possible directories:
  locations:
    - type: url
       target: https://storage.cloud.google.com/my-bucket/some-folder/*
    - type: url
       target: https://storage.cloud.google.com/my-bucket/some-folder/child-folder-1/*
    - type: url
       target: https://storage.cloud.google.com/my-bucket/some-folder/child-folder-2/*

I just wanted to list a location of a single bucket and for backstage to just chomp up any backstage related configuration as the bucket is dedicated to just backstage yaml configuration.

authors: Rugvip freben lucasteligioridis

repo: backstage/backstage