text:
java.sql.SQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes

java.sql.SQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes
1071
something wrong when init database
database connect success
load BurpDataCollector success !
add data to insert queue finish
java.sql.SQLSyntaxErrorException: Table 'data_collect.host_full_path_map' doesn't exist
export finish!
Scheduled export execution completed
java.sql.SQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes
1071
something wrong when init database
java.sql.SQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes
1071
something wrong when init database
add data to insert queue finish
java.sql.SQLSyntaxErrorException: Table 'data_collect.host_sub_map' doesn't exist
export finish!
Scheduled export execution completed
add data to insert queue finish
export finish!
Scheduled export execution completed
add data to insert queue finish
export finish!
Scheduled export execution completed
add data to insert queue finish
export finish!
Scheduled export execution completed
add data to insert queue finish
export finish!
Scheduled export execution completed
add data to insert queue finish
export finish!
Scheduled export execution completed
add data to insert queue finish
export finish!
Scheduled export execution completed
add data to insert queue finish
export finish!
Scheduled export execution completed
add data to insert queue finish
export finish!
Scheduled export execution completed
add data to insert queue finish
export finish!
Scheduled export execution completed
add data to insert queue finish
export finish!
Scheduled export execution completed
add data to insert queue finish
export finish!
Scheduled export execution completed
add data to insert queue finish
export finish!
Scheduled export execution completed
add data to insert queue finish
export finish!
Scheduled export execution completed
add data to insert queue finish
export finish!
Scheduled export execution completed
add data to insert queue finish
export finish!
Scheduled export execution completed
add data to insert queue finish
export finish!
Scheduled export execution completed
add data to insert queue finish
export finish!
Scheduled export execution completed
java.sql.SQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes
1071
something wrong when init database
add data to insert queue finish
export finish!
Scheduled export execution completed
加载该插件错误提示

换成mysql 5.5 以上版本

authors: QdghJ blockchainguard


text:
pikatests 已经不可用？

我根据 TESTREAD 中的描述准备好之后，执行
tclsh tests/test_helper.tcl set

会提示
Cant' start the Redis server
...
ERROR:Please specify the conf file path

我确定 default.conf 已经存在于 ./tests/assets/ 目录下了

@jojo59516 pikatests是可用的, 因为多了一级目录, 你可以按照redis的测试方法将pikatests下tests文件夹移动到pika的根目录下,然后取消server.tcl第221行代码的注释, 启动方式修改为-c的.

@Leviathan1995 赞👍

source: gharchive/issue

license_type: permissive

license_source: github-api


text:
info命令应该带上redis_version信息，否则springboot框架健康监测异常

info命令的返回包中没有“redis_version:xxx”信息。
pika_version:3.3.5
pika_git_sha:16283243d4d8e87668549d632d6cd13c8ac94016

对于springboot框架依赖于redis_version字段，缺了就报异常导致启动不了

{"status":"DOWN","details":{"rabbit":{"status":"UP","details":{"version":"3.3.5"}},"diskSpace":{"status":"UP","details":{"total":105554128896,"free":39503593472,"threshold":10485760}},"refreshScope":{"status":"UP"},"hystrix":{"status":"UP"},"redis":{"status":"DOWN","details":{"error":"java.lang.IllegalArgumentException: Value must not be null"}}}}

从逻辑上讲，pika的info信息里不应该包含redis的版本信息。
1 可以尝试把spring boot框架的宏 REDIS_VERSION 赋值为pika_version。
2 定制pika。尝试修改下pika的info信息输出，把pika_version 改为redis_version.

修改springboot这种标准的全家桶源码不太方便，不过可以自行重载实现redis健康检查。只是无法做到无缝把redis替换为pika。


text:
QML course content feedback

For the dataset representation in the computational basis, in the Amplitude encoding section, summation should goes from i=1 to NxM if the alpha amplitude should represent each feature in the dataset. The state |psi (x)> represent the feature(s) of one sample, while the dataset should represent the entire set of features and samples, which have dimension N x M. Actually, this is written in text below the summation, but summation goes from 1 to N, instead from 1 to N x M.

This repository is no longer maintained. It has been superseded by IBM Quantum Learning.

authors: JRussellHuffman vabarbosa

repo: Qiskit/platypus


text:
QuantumKernelTrainer class optimizer not supporting bounds options.

Environment

Qiskit Machine Learning version: 0.5.0
Python version: 3.9.15
Operating system: Linux

What is happening?
The default optimizer of the QuantumKernelTrainer class is SPSA, which is a free constraint optimizer. However, if another QisKit class optimizer which supports constraints is used such as SLSQP or L_BFGS_B, the code of the mentioned class doens't support this options because of how the optimizer is called.
Notice that the following lines from the fitmethod of the QuantumKernelTrainer class, there is no bounds argument included in the call of the minimize routine.
if callable(self._optimizer):
    opt_results = self._optimizer(fun=loss_function, x0=self._initial_point)
else:
    opt_results = self._optimizer.minimize(
        fun=loss_function,
        x0=self._initial_point,
    )

https://github.com/Qiskit/qiskit-machine-learning/blob/5abda6c0fd303453229f9caabcd831de792fdd75/qiskit_machine_learning/kernels/algorithms/quantum_kernel_trainer.py#LL212-L218C14
Therefore, submiting an optimizer with the bounds argument results in the following error
File .../python3.9/site-packages/qiskit/algorithms/optimizers/scipy_optimizer.py:148,
in SciPyOptimizer.minimize(self, fun, x0, jac, bounds)
    145     swapped_deprecated_args = True
    146     self._options["maxfun"] = self._options.pop("maxiter")
--> 148 raw_result = minimize(
    149     fun=fun,
    150     x0=x0,
    151     method=self._method,
    152     jac=jac,
    153     bounds=bounds,
    154     options=self._options,
    155     **self._kwargs,
    156 )
    157 if swapped_deprecated_args:
    158     self._options["maxiter"] = self._options.pop("maxfun")

TypeError: scipy.optimize._minimize.minimize() got multiple values for keyword argument 'bounds'

How can we reproduce the issue?
Here is a short small example that prompts the error:
from qiskit.circuit.library import TwoLocal
from qiskit_machine_learning.kernels import TrainableFidelityQuantumKernel
from qiskit_machine_learning.kernels.algorithms import QuantumKernelTrainer
from qiskit.algorithms.optimizers import SLSQP
from qiskit.utils import algorithm_globals
import numpy as np

# Define a feature_map (8 total parameters, first 4 to encode data, last 4 to train)
ansatz = TwoLocal(
    num_qubits=2,
    rotation_blocks=['rz','ry'],
    entanglement_blocks='cx',
    reps=1)

# Random data set (4 features, 5 items)
rand = algorithm_globals.random.uniform
x_train = rand(0,2*np.pi,(5,4))
y_train = np.heaviside(rand(-1,1,5),0)

# Define a trainable kernel (last 4 parameters for training)
TFQK = TrainableFidelityQuantumKernel(
    feature_map = ansatz,
    training_parameters = ansatz.parameters[4:8]
)

# Define quantum kernel trainer
QKT = QuantumKernelTrainer(
    quantum_kernel=TFQK,
    optimizer=SLSQP(maxiter=1000,options={'bounds':[(-np.pi,np.pi)]})
    # SLSQP(maxiter=1000,bounds=[(-np.pi,np.pi)]) would also prompt the error
)

# Fit the quantum kernel
# This line prompts the error
QKT.fit(x_train,y_train)

What should happen?
You would expect the fit process to run properly, since the option bounds is supported by the optimizer QisKit class.
Any suggestions?
Since bounds argument is not working, I guess that jac argument won't work. In the case of the latter, I think it would be rarely used, but I think it would be right to include it, for completeness.
With the last git version (but not the current qiskit_machine_learning release), this error can be patched by defining a custom optimziation function as follows, for example for maxiter=1000 and bounds=[(-np.pi,np.pi)], for the SLSQPoptimizer:
def custom_optimizer(fun,x0):
    return SLSQP(maxiter=1000).minimize(fun=fun,x0=x0,bounds=[(-np.pi,np.pi)])

One alternative solution for the bounds would be to use the validate_bounds class from qiskit.utils which retrieves the bounds from the ansatz (in this case the feature_map).
I would like to work on this issue if possible, so I would like to be assigned to it!

Thanks for raising this issue. I think the problem is broader rather than just in QuantumKernelTrainer. While the proposed solution closes the local problem in QKT the same issue will persist in quantum neural models as well. So, the solution must fix all similar problems, not only in QKT. There's an issue opened to discuss the problem: https://github.com/Qiskit/qiskit-terra/issues/9718.

authors: adekusar-drl rlosadagarcia

repo: Qiskit/qiskit-machine-learning


text:
Fix mypy errors created with latest numpy

Summary
numpy 1.22.0 released for python > 3.7 added mypy errors.
Since python 3.7 still uses numpy 1.21.5 the mypy.ini warn_unused_ignores was disabled until numpy 1.22.0 is available for python 3.7. This way ignore can be added on either numpy version without affecting the other.
Details and comments

Pull Request Test Coverage Report for Build 1646262915

8 of 8 (100.0%)  changed or added relevant lines in 3 files are covered.
2 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.02%) to 85.343%

Files with Coverage Reduction
New Missed Lines
%

qiskit_nature/drivers/psi4d/psi4driver.py
2
85.47%

Totals

Change from base Build 1634793320:
-0.02%

Covered Lines:
10102

Relevant Lines:
11837

💛  - Coveralls

As far as I can see the latest numpy requires Python 3.8 or above so will never support 3.7 https://github.com/numpy/numpy/blob/984a1ac8c911fa0130a6801a897642dd850796e6/setup.py#L36-L39 Are we therefore obliged to keep this as a solution now until such time that we no longer support Python 3.7 - from the EOL date for 3.7 it seems that is most likely mid next year (2023),

@woodsp-ibm I just changed the fix to use cast instead of ignore so it works in both old a new numpy without the need to change mypy.ini

source: gharchive/pull-request

authors: coveralls manoelmarques woodsp-ibm

repo: Qiskit/qiskit-nature


text:
Text drawer can't draw composite instructions

composite gates (i.e. without classical registers) work fine, but with classical registers it cannot be drawn. The logic is here:
https://github.com/Qiskit/qiskit-terra/blob/109d8ea562987b9309501b9260c80180216061eb/qiskit/visualization/text.py#L849
latex and mpl can do this.

Example code?

It this the expected output?
q = QuantumRegister(4)
c = ClassicalRegister(4)
circ1 = QuantumCircuit(q, c, name='my instruction')
circ1.measure(q, c)
inst = circ1.to_instruction()

circ2 = QuantumCircuit(q, c, name='circ2')
circ2.append(inst, q[:], c[:])
circ2.draw(output='...')

Latex

MPL

Text
Error.

No mpl and latex also need to be fixed. The box needs to extend over the classical registers as well. But at least they draw something. I would say text is priority since that's the default drawer.

repo: Qiskit/qiskit-terra


text:
Fix #7438 - Remove redundant skip in test

Summary
Fixes #7438 by removing redundant
Details and comments
This test was disabled wrongly, given that Pull Request 741 in aer was merged and newer versions of aer have been released since.

You're welcome to go through this and sort it out, but I think it might be a little more complex than just removing the skip. I had a very brief go before Christmas, and I think I hit some old code in the ConfigurableFakeBackend that was pretty out-of-date and might need a fair bit of updating. I don't remember 100%, though.

Okay, I shall look into it. Thanks for the update!

@jakelishman, would you be able to provide an insight to the most recent test fail as I wasn't really familiar with qiskit pulse.
Besides that, ConfigurableBackend had a parameter version which stood for backend_version. This was overlapping with the version parameter in the BackendV2 and thus was changed to fix previous traceback.

At @javabster's request, I've fixed the merge conflicts that were in this PR in order to have it run its CI again, but I don't know any context. My only comment above was just that I'd attempted to remove the unittest.skip line before and seen some failures (and sorry that I never replied to the question at the time).
It's possible that #9168 could be used instead of this PR, but I don't know the context of either PR, nor of the code in question here.

Closing in favor #9168 . Thanks!


text:
Add dummy extra for CSPLayout

Summary
This currently has no effect, but since we no longer use CSPLayout in
the preset pass managers, we are moving to demote its dependency
python-constraint from a requirement to an optional.  This is the
preliminary step of the process, which allows packages to depend on
qiskit-terra[csp-layout-pass] before we actually make the switch.

Details and comments
Supersedes #7733 for Terra 0.20, but #7733 will be the follow-on to this PR.

Pull Request Test Coverage Report for Build 2064847374

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 83.757%

Totals

Change from base Build 2064574046:
0.0%

Covered Lines:
53163

Relevant Lines:
63473

💛  - Coveralls

authors: coveralls jakelishman


text:
Pulse Compiler and IR

Goal:
Introducing a modern compiler for pulse module and dedicated IR for it.
RFC:
https://github.com/Qiskit/RFCs/blob/master/0012-Pulse-Compiler-and-IR.md

[ ] Logical elements and frames #10694
[ ] PulseIR
[ ] Support logical element and frame representation in pulse instructions
[ ] Pulse compiler
[ ] Compilation passes
[ ] Re-routing existing compilation tasks to the compiler

I assume this should remain open for the other tasks to be completed

Thanks Eli!

repo: Qiskit/qiskit


text:
小建议

1.入库通知背景 可以联动radarr & sonarr 下的/meida/cover 文件吗 ？  这样看上去相关性更强一点（花里胡哨的）。

2.虽然对我来说影响不大，但是UTC和GMT时间问题 可能让一部份强迫症难受。

xarr家族停用了，不适用我看老剧和国漫的习惯，再一个我是PLEX用户，emby是群友们催逼着搞的，大概率不想增加这个
关于时间，docker的话添加一个环境变量的参数即可：TZ=CST-8

license: MIT


text:
Fix cv2 mypy errors in pipeline

A new release of opencv-python caused mypy to fail in the pipeline.
These changes should fix the pipeline and should also work with the old version of opencv-python.

@eendebakpt I've briefly looked into fixing the mypy issues, but this does not seem trivial in this case, so I ignored them. I also looked into the issue with coveralls. It seems the issue with coveralls is that the repository on coveralls was created with a different account than you are using now (peendebak) which no longer is a member of the QuTech-Delft group on Github. The options to fix this are:

Delete the repository on coveralls and recreate it with a new account. This should fix the issue but means we loose historical data.
Contact coveralls support and ask them to change the owner.

But I am not sure how important it is to you that this check remains working on this repo since from what I've heard no new features are being actively developed QTT. For now the check is no longer required so MRs can be merged without it.

repo: QuTech-Delft/qtt


text:
Create a sample react app using resaga to do CRUD

Consider those examples in here https://github.com/redux-saga/redux-saga/tree/master/examples
Might not able to mimic everything, but those with API requests should work

Try nightmare or selenium to write Acceptance Testing for redux-saga Async example.

repo: QuanDhz/resaga


text:
Make ndindex() match exceptions from NumPy

It will always raise IndexError or NotImplementedError. Previously it would
raise TypeError. The error messages also match NumPy's.
This also more accurately tests array types to see if they are integer or
boolean arrays.

Thanks!

authors: asmeurer scopatz

repo: Quansight/ndindex


text:
Calculate Running Statistics & Make Accessible In Algorithm

Calculate the running statistics for the algorithm such as sharpe / sortino etc and make their values accessible in the algorithm.
==
Expected Behavior
Be able to access runtime statistics (e.g. win rate, sharpe ratio, etc) with the algorithm.
Actual Behavior
Algorithm statistics are calculated at the result handlers and users don't have access to this information during an algorithm execution which may be important for live algorithms.
Link: https://www.quantconnect.com/forum/discussion/3619/better-live-tracking-statistics/p1
Potential Solution
N/A
Checklist

[x] I have completely filled out this template
[x] I have confirmed that this issue exists on the current master branch
[x] I have confirmed that this is not a duplicate issue by searching issues

Yes! I can do that from the algorithmic trading platform AlgoTerminal, www.algoterminal.com
Please add that to QuantConnect.

Is that possible to do right now?

Create IStatisticsService Interface, make IResultHandler implement statistics service.
Add IStatisticsService.StatisticsResults method no args; returns StatisticsResults instance.
In SetAlgorithm() set the IStatisticsService to the algorithm as a private algorithm.SetStatisticsService(_statisticService);
In IAlgorithm create property Statistics return StatisticResult object
in QCAlgorithm implement Property as read-only get { return _statisticService.StatisticsResults(); }

Bonus points if:

If we can avoid recalculating it at the end of the backtest and reuse the same object
Add custom entries to the dictionary of result statistics. Statistics.SetSummaryStatistic(key, value)
Which together might open the possibility to add custom properties to the result summary

im not sure if these stats are written out into the backtest results json, I didn't see them. They are available in the backtest though. It would be great if they were added to the written out json files on a per period basis for example:
        for symbol, trade_bar in slice.Bars.items():  # there is 1 tradebar for each symbol
            tradeBar: TradeBar = trade_bar
            holding: SecurityHolding = self.Portfolio[symbol]

            ticker: str = holding.Symbol.Value

            self.statsHistory.securityStats[ticker].profits.append([timestamp, holding.Profit])
            self.statsHistory.securityStats[ticker].holdingsValues.append([timestamp, holding.HoldingsValue])
            self.statsHistory.securityStats[ticker].unrealizedProfits.append([timestamp, holding.UnrealizedProfit])
            self.statsHistory.securityStats[ticker].unrealizedProfitPercent.append([timestamp, holding.UnrealizedProfitPercent])
            self.statsHistory.securityStats[ticker].averagePrices.append([timestamp, holding.AveragePrice])
            self.statsHistory.securityStats[ticker].quantitys.append([timestamp, holding.Quantity])
            self.statsHistory.securityStats[ticker].isLongs.append([timestamp, holding.IsLong])
            self.statsHistory.securityStats[ticker].isShorts.append([timestamp, holding.IsShort])

also it would be great if we could download the files from cloud in the same format maybe via a zip as they get generated locally. cloud has orders in a csv vs json for local for example. a few other differences.

authors: JackMMMC ari62 jaredbroad voidale

repo: QuantConnect/Lean


text:
Options Support

Support for options trading. Broadly speaking this will have several pieces. Lets scope out the depth of the changes needed here:
PHASE 1:
LEAN

New security type
Options fill models
Option pricing/fee models
New option data type which holds the strike price/deltas/thetas etc for a specific option-strike-call/put-expiration..
Option symbol builder (combining all the pieces which make the option unique)
Option data type also needs to look up the option data for its symbol.
AlgoManager sends in add subscription requests to the Datafeed who adds the option data requested (and creates associated security objects)

Proposed User Facing

AddSecurity("SPX", +2(strikes assumed symetrical), Calls, TimeSpan.FromMonths(6))
User adds an Underlying, +- strike deltas from market, type, and TimeSpan in future.
We access the option data in the slice array, simplify indexing:
slice.Options["SPX", 2 (strikes), OptionType.Call].ToList() of Option objects.

ToolBox:

Tool for converting from X-popular data provider into LEAN data format.

PHASE 2:
Brokerage and live trading support. TBD.

Mentioned in this forum post: https://www.quantconnect.com/forum/discussion/769/work-with-quantconnect-options-expert-required

I can't call me an options expert or trader, because I am neither, but since I am a [CQF alumni] (http://www.cqf.com/Alumni/graduates), I think I can take care of phase 1.

:) cool, lets scope out the design. Im sure there's things I've forgotten --- what else needs to be written to support options? Step by step -- from parsing the data, storing the data, creating objects... I think the work will touch all of LEAN at different levels.

I think that parsing and storing the data would be similar to equities.
The objects would have the members that you mentioned and the model to calculate it. I think we have to use "vanilla" Black-Scholes to get implied volatility and the greeks as a default and the user could implement his/her own model replacing Black-Scholes if he/she wants to.
We could implement options classical strategies such as straddle, butterflies, etc... too so that a user could just write Straddle("APPL", 2 strikes or sigma).

I have worked with Options Trading.
If you guys need some help I'd be happy to contribute.
I think QuantLib has two great lists: one for pricing models and one for financial instruments that could be used as reference.

Data
Step 1: Tool for converting from X-popular data provider into LEAN data format.
Step 2: Storing the Data.
Step 3: Parsing the Data
Concerning the storage, the folder with tick/second/minute/hour files must include the underlying name, type (call or put), style (european or american), strike and expiration month and year. Or the folder only include the underlying name and the files hold the rest of the information. Or, we could add more subfolders: Data/options/usa/underlying/type/style/expirationyear_expirationmonth/strike_yyyyMMdd.zip.
We need map files to account changes in the strike due to dividend payments.

Good idea @AlexCatarino: lets start piece by piece -- creating and parsing the data is a good first step. Thanks @guarilha for the offier, please contribute to the thread! :)
Does the style vary within a market? (e.g. USA is all non-exercisable) are there both styles in European markets? I think we need the full date to handle weekly options as well
The strike is sometimes fractional right - so it would be price of the strike in centi-cents (1/10000 of a dollar like our equities): expiration YYMMDD, then price of the asset over time in the final zip. e.g. $17.5 strike
Also finally it has to be trades and quotes: and to make it the same format as the other files lets flip the date:
Data/options/usa/spx/put/exercise/20140801/yyyyMMdd_1750000_trade.zip

Tagging some others who've registered interest in options: @rbohac @cuda @craig-stevenson

Wow Super, we are going to get Options support. It would be nice on the top-level to have some abstract order combos like Spreads, Iron-Condor, Calendar.
It makes a bit hard to code the execution model but it is really a good help when writing the algo.

@jaredbroad According to the CBOE, many index options are European-style. I couldn't find any info that would tell me whether weekly options are all American or European-style.
So far, I got the following format to handle all kinds of options:

Data/options/usa/ underlying / timespan / type / style / expirationdate / yyyyMMdd_1750000_trade.zip
where timespan is the string "monthly" or "weekly". I don't think that full date is enough to distinguish monthly and weekly options, since CBOE itself uses a "W" in the ticker for weeklys and Fridays are not reserved for weeklys to expire.

Are we going to put together in the same "options" folder equity options, fx options, commodity options, etc?

Great to know US has both styles,
I believe there's still only one "type" per date, " (i.e, Weeklys are not listed if they would expire on a 3rd Friday or if a Quarterly option would expire on the same day)." -- does this mean if the Weekly and Monthly line up, the monthly wins. I don't think there really is a "type" its just a date the option sold for, is there some page where we can see two identical options with same date?

"Are we going to put together in the same "options" folder equity options, fx options, commodity options, etc?" -- Good question, do you know the variation of the namespace for the different types? What format are each name?
Whether we split them up physically in data should depend on whether the Option object which is generated is different across each sub-option-class. i.e. What extra information will each option have? Is it enough to have an option base class, with CommodityOption extension?

I thought it would be handy to throw some possible use cases on here:
Short Put & Vertical use case

I'll commonly want to enumerate expiration dates for an underlying. As an example I'll want to find a monthly expiration nearest to 45 days out.
I'll commonly want to select strike prices based on the greeks. As an example, I may look for a PUT with nearest to a delta of -0.05
From number 2 I'd want to move up or down the option chain based on the open interest.

Covered Call Use Case:
Using technical analysis I'll select an underlying and want to manage covered call writing for cost basis reduction. This scenario is a little easier:

Enumerate expiration looking for an appropriate expiration based on a calculation
Ability to enumerate strikes finding strikes that are nearest to a goal with enough open interest and/or a bid/ask spread within a calculated range (a helper property for midpoint would be handy)
Ability to easily compare a call price against a price for the same strike on a different expiration (compare shorter weekly to monthly for example)

Note: Strike price availability varies by expiration. For example: WYNN -> Oct15 offers in $1 increments whereas OCT215 (weekly) offers in $0.50 increments. We will need to have the concept of "Not Offered".
Would be handy to have functions like (in Psuedo-code):
Securities[symbol].GetMonthlyOptionChain(12).GetStrike(1200).Put.Midpoint

@jaredbroad
I just read specifications here and I understand that all is fine, we don't need to make an explicit distinction between monthly or weekly or quarterly options which is wonderful because it would imply different pricing models.
The difference between options for different underlying security is the cost-of-carry. We need to inform the dividend yield to compute it for equity options and risk-free rate for the foreign currency for FX. So we don't need to split them physically and one option class is enough. If we split it, it would be only a matter of organization, so that's up to you.
Very handy, @rbohac. Thanks. You would like functions like GetMonthlyOptionChain(12) and GetWeeklyOptionChain(12) to make a distinction between both and maybe GetAllOptionChain(12) to be able to trade monthly and weeklys indifferently?

@AlexCatarino @rbohac  - can you explain the importance of the distinction between the weekly/monthly/quarterly contracts? My understanding is that  the weeklys are just offered closer to the expiry, so CBOE states that weekly contracts are available for a consecutive six weeks.
Are there benefits to trading one verse the other? I would imagine the open interest would be higher in the monthly/quarterly since they've been offered for longer. I just want to understand the desire to segregate them in the algorithm.

Hey, @mchandschuh! Fact is that I've never looked closed into weeklys. I'm learning here that there's no actual difference from the quantitative finance perspective. I hope other could tell us if there is a difference from the options trading perspective (apart of liquidity and open interest, of course).

Weeklys are actually quite liquid in some names like SPY and QQQ.  Take a look at the open interest and the volume.  They are traded primarily on the nearest few weeks to the extent that demand supports it, and they are priced just like any other option.
Weeklies are popular with hedgers, but have also become quite popular with premium sellers who want to have steeper theta decay in their portfolio as much as possible, which is well suited to a weekly product vs a monthly/quarterly cycle product.
Someone may have already said this, but in regards to the option symbols and the chain, there is no difference in the option symbol nomenclature for weeklies vs monthlies vs quarterlies, just the date is different.  And the regular monthly/quarterly  options end up being the weekly option of the week that they expire in, so there is no dual listing/valuation issue to be concerned with.
HTH
-Ashley.

On Sep 17, 2015, at 15:40, Alexandre Catarino notifications@github.com wrote:
Hey, @mchandschuh! Fact is that I've never looked closed into weeklys. I'm learning here that there's no actual difference from the quantitative finance perspective. I hope other could tell us if there is a difference from the options trading perspective (apart of liquidity and open interest, of course).
—
Reply to this email directly or view it on GitHub.

This is fantastic. I just stumbled on QuantConnect a couple of days ago and was disappointed to see options weren't supported yet. It's great to see some action here as I haven't found anyone else doing anything around options.
A couple of thoughts on the discussion so far:
Weekly Options:
As @glyphard pointed out I find them useful for premium selling. There can be issues with liquidity and you often need to adjust your strikes when rolling from weeklies to monthlies. That's only a problem for execution.
The only issue I see is marking weekly and quarterly options so you can screen them in or out. I like the (W), (R) or (M), (Q) format you see on some platforms.
A couple ways I often see expiration dates being quoted are SEP 4(W), SEP 11(W), SEP 18, SEP 25(W), SEP 30(Q) or SEP1, SEP2, SEP3, SEP4, SEP5. I find the former much more clear.

Welcome @jaymjordan! :)
@all,
In general its important to separate engine concerns from API concerns. I think we're all in agreement there's no physical difference in the data between a weekly/monthly/quarterly option. So for the purposes of the engine we'll treat the data as the same. We can separate them in the API once we get there.
Now we've roughly figured out the data structure, I think its important to do this in tiny tiny steps to make sure we can actually finish and keep the reviews manageable.
So, first mini-project: Create a QuantConnect.ToolBox project which converts options data from a popular vendor into LEAN format discussed above. @AlexCatarino can you spearhead this one? Which vendor should we start with?

@jaredbroad Yes, I can take the data conversion.
By the way, I'll be moving my GitHub project on converting Bovespa data into LEAN format.
I don't know about vendors. You do the shopping, I cook the dinner. ;-)

@AlexCatarino - excellent, I've chatted with a few vendors and will pick one and get a sample file to begin the conversion.

OK after chatting with some vendors we might need to process the data to get it into our format, something we can use for options backtesting. What do you guys think about this data: 1 Minute Aggregations: With Values of:
Time, Underlying, Strike, Expiration, Trade(Open, High, Low, Close), Quote(Open(Bid,Ask), High(Bid,Ask), Low(Bid,Ask), Close(Bid, Ask)), TradeVolume
If there were no trades in a minute we'd just have an updated quote range bar and 0-volume on the final column.
Would this be enough to correctly do custom IV models?
With the close bid-ask we can at least model the fill prices accurately.

You also need to include the option type( e.g.  put or call). IB's API call's this property 'Right' but that's always been a confusing property name, IMHO...  So I suggest choosing something more descriptive, like 'OptionType'.

On Sep 23, 2015, at 16:58, Jared notifications@github.com wrote:
OK after chatting with some vendors we might need to process the data to get it into our format, something we can use for options backtesting. What do you guys think about this data: 1 Minute Aggregations: With Values of:
Time, Underlying, Strike, Expiration, Trade(Open, High, Low, Close), Quote(Open(Bid,Ask), High(Bid,Ask), Low(Bid,Ask), Close(Bid, Ask)), TradeVolume
If there were no trades in a minute we'd just have an updated quote range bar and 0-volume on the final column.
Would this be enough to correctly do custom IV models?
With the close bid-ask we can at least model the fill prices accurately.
—
Reply to this email directly or view it on GitHub.

I have finished a first version of the data converter where we read the data from a vendor and write it on Lean format. Next, we will feed Lean with this data.
I've noticed that the Tick and TradeBar object need to be extended to incorporate options information (strike price, expiration date, call or put, etc). Also I think that Tick and TradeBar are not quite ready to accept quote data. For example, tick don't have any field for quantity of bid and ask.
It'd be odd to out quote data in an object called TradeBar, but I don't know if we want to rename it or create a QuoteBar object.

This is a tricky question! It covers virtually all aspects of data + options and future proofing! :) I had a brain storming session with Michael and this is what we sketched up so far:
0> Adding BidSize and AskSize to the Tick type will allow it to support all the tick information required.
1> Like Equity and Forex, Option type can be the security:
Option : Security
3> Add an IContract type to all BaseData as well. This would be part of the Subscription for the security object and we'd set a readonly reference to it in the BaseData.
4> Then OptionContract is a type of IContract, which holds expiry, strike, deltas, greeks etc. for the Tick/Bar specific to options.
4.5> TradeBar changed to inherit from IBar. TradeBar : IBar {} -- IBar would define the Time-OHLC comonents of the bar. TradeBar would add in the Volume.
4.6> QuoteBar : IBar, where the OHLC is the midpoints of bid and ask.  Then QuoteBar.Bid and QuoteBar.Ask are also IBar properties which are the OHLC of the bid and ask prices. They could be accessed by quote.Ask.High
5> In the same way as Ticks, TradeBar/QuoteBar types could have an IContract field which specifies the option contract we're talking about (IContract lives on the BaseData object). So data (Ticks, QuoteBar and TradeBar etc) all have contracts.
5.5> We create an EquityContract and ForexContract which might hold other unknown information specific to the equity/forex. e.g. Shortable, Counterparty etc
OptionContract : IContract {
TradeBar, Expiry, Strike, Delta, Greeks (Derived values)
}
6> Like Ticks and TradeBars type, Options type could hold collections of chains indexed by symbol so users can use  OnData(Options data) {  data['GOOG']  }
Options : DataDictionary<Symbol, OptionChain>
7> Inside the OptionChain collection would be all the strikes and expiration for a symbol at a point in time, along with a lot of helper methods for accessing / searching the data:
OptionChain {
Time, Symbol, BarCollection (TradeBars and QuoteBars)
//All the helpers -- get X . get Y . Call .
Get(strikes: -2, expiry: TimeSpan.FromWeeks(2));

}
Feedback welcome!
Other thoughts:

Make liquidation behaviour at the delisting event a pluggable setting, so then option exercising can be handled using the standard QC delisting system.

Each option-strike-expiry would be a unique security, with a long type symbol code. This would enable actions like: Buy(symbol:"SPY.C.1950.20151101", quantity:11); The Long form symbol could have a wrapper which converts it to real numbers: Buy("SPY.C.-2Strike.14-Days", 11);

We'd need a brand new concept we don't have yet, an IExerciseModel, for converting the contract to stock. This could be set on the Option security like the fill models, and would allow for designing European vs American style exercise.

Concerning the data conversion and parsing, I don't have nothing to add to the solution @jaredbroad and @mchandschuh 's brainstorm gave us for now.
In order to change LeanDataWriter to convert the vendor data properly (without relying on alternative objects for Tick, TradeBar and QuoteBar) and parse the converted data into Lean, we need to parts of all @jaredbroad listed from 1 to 4. For this phase, OptionsContract doesn't need the greeks and implied volatility.

Jared's proposed solution makes sense to me.
The only piece that I'm unclear on is how you tell that the option you are looking at in the 'ondata' is a put or a call.   I may have missed that detail in the thread.

On Oct 21, 2015, at 12:37, Alexandre Catarino notifications@github.com wrote:
Concerning the data conversion and parsing, I don't have nothing to add to the solution @jaredbroad and @mchandschuh 's brainstorm gave us for now.
In order to change LeanDataWriter to convert the vendor data properly (without relying on alternative objects for Tick, TradeBar and QuoteBar) and parse the converted data into Lean, we need to parts of all @jaredbroad listed from 1 to 4. For this phase, OptionsContract doesn't need the greeks and implied volatility.
—
Reply to this email directly or view it on GitHub.

Hey @glyphard -- it would be specified in the OptionContract : IContract, which would live on the base data. Then the Collection of TradeBars / QuoteBars in the OptionChain for this timestep would have all the information it needs to be searched by the UX layer.

Ok, makes sense.
Thanks for clarifying.

On Oct 21, 2015, at 15:26, Jared notifications@github.com wrote:
Hey @glyphard -- it would be specified in the OptionContract : IContract, which would live on the base data. Then the Collection of TradeBars / QuoteBars in the OptionChain for this timestep would have all the information it needs to be searched by the UX layer.
—
Reply to this email directly or view it on GitHub.

Just to clarify, the addition of the IContract abstraction will be done separately. It will require some serious changes and thought to the way essential subscription information is passed around the system.
Looking at he roadmap, we should be able to skip number 3 ( add IContract) and continue with the IBar implementation. Also, work can continue on the concept of the OptionChain which should be relatively insulated from the IContract changes.
I'll try and implement the IContract system within the next week so that this doesn't turn into a roadblock.
Apologies for any confusions/miscommunication.

Something to think about. How will a user request to watch pricing updates for a particular option? We will need to have something that limits this.
For instance AddSecurity(SecurityType.Option, Symbol, Resolution.Minute,Expiration,Strike,Put);
There is too much data to continuously refresh the entire option chain for a security.
Also note that brokers (IB for sure) have limits on the number of securities you can subscribe to at any given time. If I recall their limit is 200 unless you buy extra data packs.
Bonus: It would be handy to also have a way to iterate through the option chain for a certain expiration. A use case for this would be to find a strike price where the Delta is within a certain threshold

There is an important distinction to draw here.
IB's limit is in place because their data is live, not for back testing,
and as such data has to flow out from a ticker plant in a data center to
quote terminals (TWS client) and they have to pay egress charges for the
bandwidth that their data consumes.  They place that limit on live option
quotes so that they don't have every customer asking for all strikes from
all expirys and not really using the data...  For the simple reason that
this would explode their data costs.
The situation in options backtesting is very differnt.  The data is already
present in the environment where it needs to be processed and does not have
to stream outside of the datacenter.  So having a subscription to one
strike or all strikes for an expiry in an underlying does not place a
significant overhead on a back test.  Granted there are limits to this, as
the data set for options, even sampled as 1minute bars, is still quite
large, the limit is much much higher than anything one would be subject to
in live trading platform like TWS.
Regarding the Bonus, I agree that it would be very helpful to have a way to
'discover' where the end's of the chain are, and to be able to know the
strike granularity as strikes are not evenly spaced and they evolve over
time as the particular expiry draws near.  (Probably worth discussing this
in a separate thread though...).
Super excited to see options in QC.
Keep up the great work guys!
On Wed, Oct 28, 2015 at 9:55 AM, Ray Bohac notifications@github.com wrote:

Something to think about. How will a user request to watch pricing updates
for a particular option? We will need to have something that limits this.
For instance AddSecurity(SecurityType.Option, Symbol,
Resolution.Minute,Expiration,Strike,Put);
There is too much data to continuously refresh the entire option chain for
a security.
Also note that brokers (IB for sure) have limits on the number of
securities you can subscribe to at any given time. If I recall their limit
is 200 unless you buy extra data packs.
Bonus: It would be handy to also have a way to iterate through the option
chain for a certain expiration. A use case for this would be to find a
strike price where the Delta is within a certain threshold
—
Reply to this email directly or view it on GitHub
https://github.com/QuantConnect/Lean/issues/166#issuecomment-151852614.

I'm new to programming in general and have not yet written my first strategy. The wheel just spins for me on the left of the page so I have been unable to use the platform. However, the ability to code in C# seems to have significant promise. The promise of Options support is incredibly exciting.
I have not seen anyone doing automated trading for options in as robust a way as this site appears that it may support, at least not the way that I like to think of doing them. With one platform, I did see some 'attempt' at trading automated equity options trading, but the real issue for me comes in with specifying an explicit strike price to trade. The reason, is that the trigger comes from looking at the underlying, where to trade an option, an explicit strike price must be identified in the option code.
In-The-Money (ITM) trades for instance, have a different strike price than those that are Out-of-The-Money (OTM).  It would be nice to be script something that measures the underlying, and then trades based on the number of strikes either OTM or ITM. This way, you would not have to say "closest to a delta of 50" or anything.  Rather, you could just specify the number of strikes OTM or ITM as a designator for which strike you wanted to choose at the time of your trigger.

Not sure how much farther you are through this right now, but I can add some coding grunt to help get this done.  If you have a task that is relatively contained and you'd like done, I'd be happy to contribute.

Thanks @pms1969 and all who wants to contribute.
Let me make an update, using Jared's post on October 21st: we have implemented BidSize, AskSize, IBar/Bar and part of QuoteBar (these are points 0, 4.5 and 4.6).
@mchandschuh is, as we speak, finishing IContract up. After this is done, we can code OptionsContract and have all the information we need these to code the app that will write the vendors' data in Lean format. That would cover all points, but 6 and 7.
Once we get there, we could talk about new assignments.

Thank you Paul! We'd welcome your help and should make headway/ship the
OptionsContract type this week. We've been doing some tidy up on data and
universe selection :) As always thanks to @AlexCatarino & all the
contributors!
The "SID" / "Symbol" class will be our new way of uniquely identifying
securities inside the engine (vs strings). The new "SID" system can hold
all the option type data in a single symbol object, check it out here:
https://github.com/QuantConnect/Lean/blob/sid/Common/Symbol.cs
https://github.com/QuantConnect/Lean/blob/sid/Common/SecurityIdentifier.cs
With SID there's a chance we won't even need the OptionContract type
anymore, we'll need to think about this to double check its still required.
If possible,@pms1969 https://github.com/pms1969 @AlexCatarino please
review this new code/branch and make sure we havent overlooked anything!
This would be a great starting contribution :) Please put any comments on
the relevant files,
On Sat, Nov 21, 2015 at 2:44 PM, Alexandre Catarino <
notifications@github.com> wrote:

Thanks @pms1969 https://github.com/pms1969 and all who wants to
contribute.
Let me make an update, using Jared's post on October 21st: we have
implemented BidSize, AskSize, IBar/Bar and part of QuoteBar (these are
points 0, 4.5 and 4.6).
@mchandschuh https://github.com/mchandschuh is, as we speak, finishing
IContract up. After this is done, we can code OptionsContract and have
all the information we need these to code the app that will write the
vendors' data in Lean format. That would cover all points, but 6 and 7.
Once we get there, we could talk about new assignments.
—
Reply to this email directly or view it on GitHub
https://github.com/QuantConnect/Lean/issues/166#issuecomment-158677066.

--
Jared Broad

I had a look at those files (and other at the branch to learn more) and I don't think you overlooked anything. With that we have all we need to write the input files in Lean format.
Maybe we don't need the OptionContract to specify the option characteristics, but we still need a class to hold (and probably calculate) the greeks and implied volatility.

Jared,  how do I supply feedback on the 2 classes?  And are you purely after logic feedback, or are you open to architectural input as well?

Hey Paul -- all of the above! :) Symbol is an enormous and powerful change
in LEAN, allowing complete, deterministic identification of assets with any
market, security type, issue date and ticker in a single class. We rewrote
hundreds of files to support the new change. This way we can specify
options expiry date, type and strike price with the symbol class. If we've
missed something architecturally its better to know now rather than later,
To keep discussions focused please comment directly on the symbol changes
(not on this options thread).
JB
On Tue, Nov 24, 2015 at 2:10 AM, Paul Saunders notifications@github.com
wrote:

Jared, how do I supply feedback on the 2 classes? And are you purely after
logic feedback, or are you open to architectural input as well?
—
Reply to this email directly or view it on GitHub
https://github.com/QuantConnect/Lean/issues/166#issuecomment-159179205.

--
Jared Broad

Found out how to do it.  Some comments already made.  I'll look at it further tomorrow.

Very exciting!
On Nov 24, 2015 12:51 PM, "Paul Saunders" notifications@github.com wrote:

Found out how to do it. Some comments already made. I'll look at it
further tomorrow.
—
Reply to this email directly or view it on GitHub
https://github.com/QuantConnect/Lean/issues/166#issuecomment-159370494.

Is there a separate branch for the options support?  I can probably offer some advice, as I've built a few options trading systems....

@lurium -The options branch will be coming shortly, trying to get a first rev working!
@AlexCatarino, @glyphard, et all: Any ideas what kinds of input data do options models require? Ideally we can set the system up to be flexible to support as many different models as possible.

Options models need, at least, three "submodels": volatility model, interest rate model and dividends yield model.
The simplest volatility model doesn't need data: the trader would assume a constant volatility. Another volatility model (the one we will probably implement as default), takes the underlying price, the information about the option contract and its price, and yields the implied volatility. We already got these input data.
To model interest rates, we need US Treasury Bond, Libor rates or Overnight indexed swap (OIS) rates.
To model dividend yields, we need dividend data.

@AlexCatarino - does it make sense for us to provide these abstractions? For example, should we have an IOptionVolatilityModel, IInterestRateModel, IDividendYieldModel ? Or should we keep it more generally as an IOptionPriceModel and allow the model internally to use the submodels?
Or maybe it makes sense that an IOptionPriceModel would use the volatility/interest rate/div yield models?

Agree on two of the models-- Vol, and Rate.
However, instead of Divs, it might be simpler and more general to provide a
"IForwardModel," which is really what the Divs and Rates are trying to
accomplish in the case of equity and equity index options.  For
commodities, the forwards are usually derived directly from some function
of the future prices.  For Vol Futures (VX), it's a more complex
derivation, but the concept is the same -- all option pricing requires the
concept of a "forward price"
On Thu, Feb 25, 2016 at 10:21 AM, Michael notifications@github.com wrote:

@AlexCatarino https://github.com/AlexCatarino - does it make sense for
us to provide these abstractions? For example, should we have an
IOptionVolatilityModel, IInterestRateModel, IDividendYieldModel ? Or
should we keep it more generally as an IOptionPriceModel and allow the
model internally to use the submodels?
Or maybe it makes sense that an IOptionPriceModel would use the
volatility/interest rate/div yield models?
—
Reply to this email directly or view it on GitHub
https://github.com/QuantConnect/Lean/issues/166#issuecomment-188835671.

Thanks for the suggestion, @lurium!
I think we should have a IOptionPricingModel that would be internally use the submodels (IOptionVolatilityModel, IInterestRateModel and IForwardModel). If the OptionPricingModel has a constructor that takes IOptionVolatilityModel, IInterestRateModel, IForwardModel, the quant would be able to use his own proprietary model(s). That's the goal.

Sounds good.

On Feb 25, 2016, at 1:20 PM, Alexandre Catarino notifications@github.com wrote:
Thanks for the suggestion, @lurium!
I think we should have a IOptionPricingModel that would be internally use the submodels (IOptionVolatilityModel, IInterestRateModel and IForwardModel). If the OptionPricingModel has a constructor that takes IOptionVolatilityModel, IInterestRateModel, IForwardModel, the quant would be able to use his own proprietary model(s). That's the goal.
—
Reply to this email directly or view it on GitHub.

I wonder if the volatility model interface should not be named IVolatilityModel. After all, the underlying volatility will be modeled, not the option volatility.
By the way, the volatility model can live outside the derivatives world, since its output can be used as an indicator (if (vol>0.5) TrendFollow(); else MeanRevert();) and/or allocation parameter.

@AlexCatarino - good point, maybe it lives on the underlying Security object as just another model. What would a 'default' IVolatilityModel look like?

@AlexCatarino - Also, how important is running daily volume for options price models?

Not very, although if you have Open Interest, that's somewhat useful -- not
to derive theoretical values or greeks, but to understand the supply and
demand and dealer inventory considerations
On Thu, Feb 25, 2016 at 3:57 PM, Michael notifications@github.com wrote:

@AlexCatarino https://github.com/AlexCatarino - Also, how important is
running daily volume for options price models?
—
Reply to this email directly or view it on GitHub
https://github.com/QuantConnect/Lean/issues/166#issuecomment-188982812.

I agree that open interest is particularly useful along with the bid/ask spread for liquidity

IOptionPricingModel Approach makes sense to me as well.  the mechanics of
where the actual data inputs for the vol/dividend would come from are a
little less clear since they are time varying at different scales.  For
instance depending on which submodel you choose, dividends can be discrete
events, and vol changes tick by tick.  OR dividends are steady/annual or
quarterly rate based, and vol is updated daily.
In any event the choices for which approach submodel to use generally are a
function of how what the consumer of the pricing data wants to do.  In
practice what some people do is use something that's 'wrong all the time'
like the BSM model and just make a super simplifying assumption like all
non observable inputs except implied vol are zero.  Then use a numerical
scheme to back out implied vol from those zero'd input assumptions with
observed option pricecs.  This approach has a name that I don't recall off
hand, but it works fairly well for market data anytime withing the last 10
years or so, is quite common, and is simple to implement.
My suggestion is that you consider implementing this first, as a 'base'
model, then extend the api and allow 'others' to add on the more complex
things.    This should help make the options implementation more tractable.
If you need a reasonably fast BSM implementation, and don't have one, let
me know, I'm happy to write one in c#... it's about 20 minutes worth of
work, and another 15 minutes for unit tests, just make an issue and point
me at it...  happy to help if this is the way you decide to go.
On Thu, Feb 25, 2016 at 4:21 PM, Ray Bohac notifications@github.com wrote:

I agree that open interest is particularly useful along with the bid/ask
spread for liquidity
—
Reply to this email directly or view it on GitHub
https://github.com/QuantConnect/Lean/issues/166#issuecomment-188993935.

Here's some initial implementation for options!

I've been thinking about what a 'default' IVolatilityModel would look like.
Say that the 'default' would calculate Implied Volatility using Black-Scholes Model.
The volatility model would start with a seed value for sigma (volatility), then a Compute/Estimate/Fit method would call the BSM method (input parameter: double sigma, returns decimal optionPrice) until the value of sigma is close enough to the one that returns the current option price.
I hope that answers what you asked.
I think we could take @glyphard's implementation, since he says it already done. I would translate into C# the Visual Basic code we find at Espen Haug's "The Complete Guide to Option Pricing Formulas".

I might have missed some of this thread but:
Most of the options we will likely deal with on QC will be American
exercise, so BSM isn't the best choice.
What you want is a Binomial Tree for stock options and the Whaley Model for
most American commodity futures options.
Codewise the you can just iterate as you mentioned, but a nice optimization
is to use the vega (even the BSM vega as it's cheap to calculate) to do a
smarter, Newton-Raphson type iteration.
You can use BSM for European options, but if you use it for American
options, you'll often get different volatilities for calls and puts of the
same strike.
On Fri, Feb 26, 2016 at 1:39 PM, Alexandre Catarino <
notifications@github.com> wrote:

I've been thinking about what a 'default' IVolatilityModel would look
like.
Say that the 'default' would calculate Implied Volatility using
Black-Scholes Model.
The volatility model would start with a seed value for sigma (volatility),
then a Compute/Estimate/Fit method would call the BSM method (input
parameter: double sigma, returns decimal optionPrice) until the value of
sigma is close enough to the one that returns the current option price.
I hope that answers what you asked.
I think we could take @glyphard https://github.com/glyphard's
implementation, since he says it already done. I would translate into C#
the Visual Basic code we find at Espen Haug's "The Complete Guide to Option
Pricing Formulas".
—
Reply to this email directly or view it on GitHub
https://github.com/QuantConnect/Lean/issues/166#issuecomment-189417882.

@lurium - The point of the thread is mostly to seek feedback on architecture plans / designs - to ensure we fully support the various ways people want to use options. This also applies the API / how people want to access the option data.
e.g. Discussing various model implementations is important so we understand the data required to calculate these models. This way we can make sure LEAN supports as many models as possible (is as generic and extendible as possible). We also want to support international markets and all security types.

Ok, then you want something like this.
Black-Scholes: European Exercise -- Indexes SPX, VIX, weeklies at the CME
Whaley: American Exercise -- Indexes with Continuous Divs or Commodities (Gold, Oil at the CME, etc)
Binomial-Tree: American Exercise -- Stock options with discrete divs (MSFT, SPY, etc)
Essentially you have to gather or somehow discover information about both the underlying contract and the terms of the option contract to pick the proper model.
You could simplify this a little, but at the price of performance -- A Special Binomial Tree will price all of the above, it's just really slow.

I should also note that there are quite a few other option models, especially for fixed income and spot-forex, but these Big 3 will get you a long way.  The inputs are very similar. I could probably point the thread to some decent implementations if you guys are interested.

Just an update; options backtesting is done, we're working on getting options trading support in live mode as well and then can publish it live.

Great news.

Does that include support for futures? (On master they don't seem to be supported yet..)
When can we expect to have support for live options trading?
I'd be happy to help move this feature along. Anything I can do?

Hi @rmagen - futures aren't done yet. @quant1729 is leading the push to finish off options & futures.
We'd welcome your help! A good PR/issue would be updating the IQFeed toolbox project to subscribe to options contracts and push the BaseData packets into the IDataQueueHandler. It is one of the steps on the path to live options trading and helps LEAN support options locally.
@AnObfuscator has created a new AlgoSeekOptionConverter and we'll be crunching the data this week! We have 400TB to chew through so it might take some time :)

The code is about as good as it's going to get. I just need to finish some testing today, and I'll submit a pull request.

Its embarrassing this has taken a whole year but we're 99% complete thanks to efforts of @quant1729 on the options and futures branch. We should have this merged into LEAN soon :)

Thank you.  this is great news. I understand from my developer that there
is no bid or ask data at all. Is that accurate? Where do you guys stand on
that?
On Sep 29, 2016 6:08 PM, "Jared" notifications@github.com wrote:

Its embarrassing this has taken a whole year but we're 99% complete thanks
to efforts of @quant1729 https://github.com/quant1729 on the options
and futures branch. We should have this merged into LEAN soon :)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/QuantConnect/Lean/issues/166#issuecomment-250617586,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMeA3s3vBPxPolAHxY7bNVI3H5-uj9Bks5qvEUDgaJpZM4F-ApW
.

Since options are not liquid for options we have made QuoteBars which have enough information for accurate fills. https://github.com/QuantConnect/Lean/tree/options-and-futures

On Mon, Oct 3, 2016 at 9:03 AM, Jared notifications@github.com wrote:

Since options are not liquid for options we have made QuoteBars which have
enough information for accurate fills. https://github.com/
QuantConnect/Lean/tree/options-and-futures

We were trying to use the quantconnect platform to execute a single trade
based on the "mark" or "mid" price.  I'm not a coder but the developer said
that there was no bid or ask pricing data, so finding the current mark
price for which to place a limit order instead of a market order, did not
appear to exist.
Let's say I wanted to write two orders, one to buy to open APPL with a
limit order at the mid price, and one to sell APPL to exit at the mid
price?  How could I write that?
Ted R Penner
www.tedpenner.com
(254) 214-3013

So options support seems to be in and working (hurray!) but how does one add data for it? I couldn't figure out the format.

Options data has tradebars, quotebars and open interest files. We provide a converter for data purchased from AlgoSeek (https://github.com/QuantConnect/Lean/tree/master/ToolBox/AlgoSeekOptionsConverter). If you have other raw options data files please contribute a converter for your format! :)

Closing as feature implementation completed; and we're now in bug fixes mode.

authors: AlexCatarino AnObfuscator bobobo1618 glyphard guarilha jaredbroad jaymjordan lssilva lurium mchandschuh pms1969 rbohac rmagen tedpenner


text:
Create Parabolic SAR Indicator

Create an "ParabolicSAR" indicator class and the associated "SAR" helper method in the QCAlgorithm base class. See introduction to indicators - https://lean.quantconnect.com/docs#topic9.html
Add the associated tests to Tests project under /Indicators to confirm the Parabolic SAR values match up with the data provided in Tests/TestData/spy_parabolic_SAR.txt file.

Hey @AlexCatarino! Just wondering if there was any progress on this? If not, are you OK with me un-assigning you in case someone else wants to take a stab at it? Thanks!

Please, do assign someone else, @mchandschuh!
I'm quite busy with delayed work right now and I don't want to hold the indicators coding back.

I just added an example psar on the forums.
during testing, my test fails after 83 updates on this line...
5/15/2013 12:00:00 AM,164.98,166.45,164.91,166.12,161.7411

Closed in commit 788b598

authors: AlexCatarino chutes jaredbroad mchandschuh


text:
Beautify transaction printing in C++

We have one way to display transactions with libsolv built in tools. (it's implemented as Transaction::print(...)).
However, we might want to make it more beautiful and more specialized for conda.
Also we need to figure out how to handle [Y/n] for accepting transactions.

Done in the last commit

repo: QuantStack/mamba


text:
dtype gets implicitly converted, and buffer is copied

When we have a function taking a pyarray but pass in a numpy array of integers, the numpy array is implicitly converted to a pyarray of doubles (but this creates a copy of the buffer).
We should think about emitting a warning in this case, or make the warning user modifiable.
Or at least clearly document it.

Actually in pybind11 there is a compile-time option like so:
void f(py::array_t<double, py::array::c_style | py::array::forcecast> array);

To signify if that array should be casted or not. Maybe we could use that, or something similar?

repo: QuantStack/xtensor-python

license_source: bigquery


text:
Renaming

Renamed the following functions:

make_view => view
make_xindexview => index_view
make_xfilter => filter

That's may look less C++, but it's simpler.
Also added the documentation of xindexview that I accidentally dropped from the documentation PR.
@wolfv I added short descriptions of index views and filter views in the narrative documentation (that I accidentally forgot to add in the documentation PR), do not hesitate to submit PRs if you think it can be improved.

The example notebook probably needs to be modified to reflect the renaming.

Indeed, I've updated it and squashed it with the previous commit.

OK, We might want to release right after this gets in so that the binder (which installs xtensor from conda, and the notebook from the repo) gets a consistent state...

We got other things to fix before we can release. I guess it's not a problem if the binder state is not consistent for some days ?

It is find as soon as it is not re-rendered.

Go! go! go!

authors: JohanMabille SylvainCorlay

repo: QuantStack/xtensor


text:
Compiler Docs: Fix Quotes

Fix some quotes in the compiler docs.

Thanks!

repo: QuantStack/xwidgets


text:
Update Misty's bio

Added recent projects.

@MaldoAlberto, in the spirit of the workshop, I opened this PR to update my bio 😄

@MaldoAlberto, in the spirit of the workshop, I opened this PR to update my bio 😄

Thank you so much Misty :D

authors: MaldoAlberto Misty-W


text:
TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/24974

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/25320

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/30065

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/36009

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/36034

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/36038

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/36059

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/41435

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/101470


text:
🛑 Biken is down

In 79bab78, Biken (https://biken.quentinp.me) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Biken is back up in 08c7f94.

repo: QuentinPhilipp/uptime


text:
The wizard is taking almost all the screen because of the instructions by default

[ ] use a max width?

No solutions for now, resize the screen by yourself

Solved in 0.0.3

repo: QuentinRa/intellij-ocaml-plugin


text:
Falta detalle de caracterizaciones de las referencias

Especificaciones detalladas de las referencias como tipo de Adquisición, tipo de tejido, entre otras características.

Se ajustó las denominaciones de Tipo de Adquisicion y tipo de Producto

repo: QuestCali/QuestComercial

license: unlicense


text:
[openqasm] two-qubit gates output by "convert_to_qasm_str" function have no comma

Execute the following code.
from quri_parts.circuit import QuantumCircuit
from quri_parts.openqasm.circuit import convert_to_qasm_str

circuit = QuantumCircuit(2)
circuit.add_H_gate(0)
circuit.add_CNOT_gate(0, 1)
print(convert_to_qasm_str(circuit))

Then this code outputs OpenQASM strings.
OPENQASM 3;
include "stdgates.inc";
qubit[2] q;

h q[0];
cx q[0] q[1];

I think that there should be a comma between q[0] and q[1] in the last line.
cx q[0], q[1];

If my think is right, I will fix it.

Thank you for your report, I think you are right!

authors: kwkbtr snuffkin

repo: QunaSys/quri-parts


text:
微调运行sh finetune/finetune_ds.sh 报错

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

[X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

[X] 我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior
我希望做全量微调（8卡A100应该可以吧），所以运行sh finetune/finetune_ds.sh
以下是运行脚本：
export CUDA_DEVICE_MAX_CONNECTIONS=1
DIR=pwd
GPUS_PER_NODE=8
NNODES=1
NODE_RANK=0
MASTER_ADDR=localhost
MASTER_PORT=6001
MODEL="/home/xxxx/llm_model/qwen-7b-chat" # Set the path if you do not want to load from huggingface directly
ATTENTION: specify the path to your training data, which should be a json file consisting of a list of conversations.
See the section for finetuning in README for more information.
DATA="/home/xxxx/pyproj/Qwen-7B/finetune/dummy_data2.json"
DISTRIBUTED_ARGS="
--nproc_per_node $GPUS_PER_NODE
--nnodes $NNODES
--node_rank $NODE_RANK
--master_addr $MASTER_ADDR
--master_port $MASTER_PORT
"
torchrun $DISTRIBUTED_ARGS finetune.py
--model_name_or_path $MODEL
--data_path $DATA
--bf16 True
--output_dir output_qwen
--num_train_epochs 5
--per_device_train_batch_size 1
--per_device_eval_batch_size 1
--gradient_accumulation_steps 16
--evaluation_strategy "no"
--save_strategy "steps"
--save_steps 1000
--save_total_limit 10
--learning_rate 1e-5
--weight_decay 0.1
--adam_beta2 0.95
--warmup_ratio 0.01
--lr_scheduler_type "cosine"
--logging_steps 1
--report_to "none"
--model_max_length 2048
--gradient_checkpointing True
--lazy_preprocess True
--deepspeed finetune/ds_config_zero3.json
报错信息：
: not foundfinetune_ds.sh: 4:
: not foundfinetune_ds.sh: 10:
: not foundfinetune_ds.sh: 15:
: not foundfinetune_ds.sh: 23:
': [Errno 2] No such file or directory/bin/python: can't open file '/home/xxxx/pyproj/Qwen-7B/
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 2) local_rank: 0 (pid: 1102994) of binary: /home/xxxx/miniconda3/envs/firefly39/bin/python
Traceback (most recent call last):
File "/home/xxxx/miniconda3/envs/firefly39/bin/torchrun", line 8, in
sys.exit(main())
File "/home/xxxx/miniconda3/envs/firefly39/lib/python3.9/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 346, in wrapper
return f(*args, **kwargs)
File "/home/xxxx/miniconda3/envs/firefly39/lib/python3.9/site-packages/torch/distributed/run.py", line 762, in main
run(args)
File "/home/xxxx/miniconda3/envs/firefly39/lib/python3.9/site-packages/torch/distributed/run.py", line 753, in run
elastic_launch(
File "/home/xxxx/miniconda3/envs/firefly39/lib/python3.9/site-packages/torch/distributed/launcher/api.py", line 132, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/home/xxxx/miniconda3/envs/firefly39/lib/python3.9/site-packages/torch/distributed/launcher/api.py", line 246, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
FAILED
Failures:
<NO_OTHER_FAILURES>
Root Cause (first observed failure):
[0]:
time      : 2023-09-22_17:48:59
host      : xxxx-SSxxxxMBX-XI6-E711
rank      : 0 (local_rank: 0)
exitcode  : 2 (pid: 1102994)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
./finetune/finetune_ds.sh: 25: --model_name_or_path: not found
./finetune/finetune_ds.sh: 26: --data_path: not found
./finetune/finetune_ds.sh: 27: --bf16: not found
./finetune/finetune_ds.sh: 28: --output_dir: not found
./finetune/finetune_ds.sh: 29: --num_train_epochs: not found
./finetune/finetune_ds.sh: 30: --per_device_train_batch_size: not found
./finetune/finetune_ds.sh: 31: --per_device_eval_batch_size: not found
./finetune/finetune_ds.sh: 32: --gradient_accumulation_steps: not found
./finetune/finetune_ds.sh: 33: --evaluation_strategy: not found
./finetune/finetune_ds.sh: 34: --save_strategy: not found
./finetune/finetune_ds.sh: 35: --save_steps: not found
./finetune/finetune_ds.sh: 36: --save_total_limit: not found
./finetune/finetune_ds.sh: 37: --learning_rate: not found
./finetune/finetune_ds.sh: 38: --weight_decay: not found
./finetune/finetune_ds.sh: 39: --adam_beta2: not found
./finetune/finetune_ds.sh: 40: --warmup_ratio: not found
./finetune/finetune_ds.sh: 41: --lr_scheduler_type: not found
./finetune/finetune_ds.sh: 42: --logging_steps: not found
./finetune/finetune_ds.sh: 43: --report_to: not found
./finetune/finetune_ds.sh: 44: --model_max_length: not found
./finetune/finetune_ds.sh: 45: --gradient_checkpointing: not found
./finetune/finetune_ds.sh: 46: --lazy_preprocess: not found
./finetune/finetune_ds.sh: 47: --deepspeed: not found
期望行为 | Expected Behavior
No response
复现方法 | Steps To Reproduce
No response
运行环境 | Environment
- OS:
- Python:
- Transformers:
- PyTorch:
- CUDA (`python -c 'import torch; print(torch.version.cuda)'`):

备注 | Anything else?
No response

I've encountered the same problem and I solved it by changing CRLF in finetune_ds.sh is with LF.

bash finetune_ds.sh 试试

前面comments有提到，linux下的shell不认\r\n，会报类似的错误。finetune_ds.sh里原本都是\n的，建议检查看看。

repo: QwenLM/Qwen


text:
Directions not available specific to Confluent

Your connector is recommended on the Confluent website, but the directions provided for implementing are not specific to Confluent.
We are continually getting the following error and can find no support for it anywhere.
[2018-02-21 17:58:51,663] INFO Started @190606ms (org.eclipse.jetty.server.Server:379)
[2018-02-21 17:58:51,664] INFO REST server listening at http://127.0.1.1:8083/, advertising URL http://127.0.1.1:8083/ (org.apache.kafka.connect.runtime.rest.RestServer:150)
[2018-02-21 17:58:51,667] INFO Kafka Connect started (org.apache.kafka.connect.runtime.Connect:55)
[2018-02-21 17:58:51,685] ERROR Failed to create job for /home/username***/MongoDb-Sink-Connector/sink.properties (org.apache.kafka.connect.cli.ConnectStandalone:97)
[2018-02-21 17:58:51,690] ERROR Stopping after connector error (org.apache.kafka.connect.cli.ConnectStandalone:108)
Can anyone help?  We are connecting to another server for our MongoDB database.
Kafka consumer configuration
name=kafka-connector-mongodb-sink
Kafka connector configuration
connector.class=org.radarcns.connect.mongodb.MongoDbSinkConnector
tasks.max=1
Topics that will be consumed
topics=avrotest
MongoDB server
mongo.host=192.168.1.224
mongo.port=27017
MongoDB configuration
mongo.username=admin
mongo.password=******
mongo.database=OurDatabase
Collection name for putting data into the MongoDB database. The {$topic} token will be replaced
by the Kafka topic name.
#mongo.collection.format=avrotest
Factory class to do the actual record conversion
record.converter.class=org.radarcns.connect.mongodb.serialization.RecordConverterFactory

Nevermind.  The JAR files need to go in a different location than that specified in your documentation for when using Confluent.  Should go in /etc/

repo: RADAR-base/MongoDb-Sink-Connector


text:
B4 - Payments and transactions

Strana 86 specifikacije. Napraviti prvo model za transakcije jer one trebaju za menjacnicu.

Implementirati kontrolere + logiku za : Novo plaćanje, prenos, primaoce placanja (3 operacije) i pregled placanja. Verifikacija ako stignete


text:
Use Hydra vocab for paging info

See https://www.hydra-cg.com/spec/latest/core/ & http://www.hydra-cg.com/.
Must decide on Linked Data Platform norms v. current setup v. Hydra for paging etc.

Another spec to consider: the collection design of TREE is designed to be compatible with both Hydra and LDP, and supports adding more semantics to what your pagination links mean (e.g., describe an ordering, or add more than just next, first, last, previous): https://w3id.org/tree/specification

Thanks @pietercolpaert! The paging semantics here have, so far been pretty bad so I think this is a good suggestion to move to the robust hydra. The HTTP paving Link headers are standard, so we can keep those.
I’ll try to fold in hydra soon: we are working on a major update to this API now which adds support for Python’s new FastAPI, not just Flask, so we can probably do all this in the same move.

authors: nicholascar pietercolpaert

repo: RDFLib/pyLDAPI


text:
[Feature request] zipalign argument

yes, i know there are other tools for zipalign. still asking will you ever bring an argument for that like -zipalign
**Describe the bug**
A clear and concise description of what the bug is.

**To Reproduce**
Steps to reproduce the behavior:
1. Used version '....'
2. Operating system '....'
3. Command '....'

**Log/Stacktrace**

**Used apk file**
Attach/share your apk

**Additional context**
Add any other context about the problem here.

All APKEditor's operation produces apk with zip aligned (4 bytes),  do you mean a feature to turn on/off zip alignment? Explain the need.

i mean this. sorry that i was not sure if it's called zipalign.
but what this basically does is reduce the apk size.
like the 737 MB apk it will become about 550 MB after optimization.
i was requesting something like that (or i have to optimize them manually afterwards)

repo: REAndroid/APKEditor


text:
Cancel deployment

Option to cancel deployment
Pending deployment would be easy, running deployments will depend on whether Symfony Process provides a way to kill a process, would need to do something like use a cache to store the cancel request and have the deployment process continually check it for a cancellation request

Work in progress #142

Work in progress #142

repo: REBELinBLUE/deployer

license: mit


text:
make all commands only slash and remove the old pm! commands

would need help making all commands slash only
I have try on the pm!new(/new) command but it break a lot of other code in different files.
will add more details later.

done fee899e12bdd2e4b6611559758cb4a436594d838


text:
🛑 CEOTR Home Page loads is down

In 2606d32, CEOTR Home Page loads (https://stg.ceotr.ca CEOTR Home) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CEOTR Home Page loads is back up in 6fbd047 after .

repo: RKTowse/upptime


text:
🛑 CEOTR Home Page loads is down

In 25278b9, CEOTR Home Page loads (https://stg.ceotr.ca CEOTR Home) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CEOTR Home Page loads is back up in e20cc4b after 2 minutes.


text:
🛑 CEOTR Home Page loads is down

In a5ac48a, CEOTR Home Page loads (https://stg.ceotr.ca CEOTR Home) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CEOTR Home Page loads is back up in 8b4e01d after 11 minutes.


text:
🛑 CEOTR Home Page loads is down

In 1ca95bd, CEOTR Home Page loads (https://stg.ceotr.ca CEOTR Home) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CEOTR Home Page loads is back up in e1a56be after 2 minutes.


text:
🛑 Sensor Tracker login is down

In 614c209, Sensor Tracker login (https://stg.ceotr.ca/sensor_tracker) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Sensor tracker login is back up in e2ddf7f after 1 minute.


text:
🛑 Sensor Tracker login is down

In 8a849f1, Sensor Tracker login (https://stg.ceotr.ca/sensor_tracker) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Sensor tracker login is back up in 52a2215 after 3 minutes.


text:
🛑 CEOTR Home Page loads is down

In b8d05c1, CEOTR Home Page loads (https://stg.ceotr.ca CEOTR Home) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CEOTR Home Page loads is back up in 5f2c16f after 2 minutes.


text:
🛑 CEOTR Home Page loads is down

In 05d881f, CEOTR Home Page loads (https://stg.ceotr.ca CEOTR Home) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CEOTR Home Page loads is back up in c400fde after 2 minutes.


text:
🛑 Sensor Tracker login is down

In dd75a63, Sensor Tracker login (https://stg.ceotr.ca/sensor_tracker) was down:

HTTP code: 502
Response time: 52 ms

Resolved: Sensor tracker login is back up in d02c19c.


text:
🛑 Sensor Tracker login is down

In 364a5f9, Sensor Tracker login (https://stg.ceotr.ca/sensor_tracker) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Sensor tracker login is back up in 6faf2b2 after .


text:
🛑 Sensor Tracker login is down

In 1a54e34, Sensor Tracker login (https://stg.ceotr.ca/sensor_tracker) was down:

HTTP code: 502
Response time: 47 ms

Resolved: Sensor tracker login is back up in 161c0d7.


text:
🛑 Sensor Tracker login is down

In 7a12586, Sensor Tracker login (https://stg.ceotr.ca/sensor_tracker) was down:

HTTP code: 502
Response time: 42 ms

Resolved: Sensor tracker login is back up in 44f1731.


text:
🛑 CEOTR Home Page loads is down

In 8a73535, CEOTR Home Page loads (https://stg.ceotr.ca CEOTR Home) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CEOTR Home Page loads is back up in 8db491e.


text:
🛑 Sensor Tracker login is down

In bc86c49, Sensor Tracker login (https://stg.ceotr.ca/sensor_tracker) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Sensor tracker login is back up in 5240f09 after .


text:
🛑 CEOTR Home Page loads is down

In 90a5d09, CEOTR Home Page loads (https://stg.ceotr.ca CEOTR Home) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CEOTR Home Page loads is back up in 5792317 after 2 minutes.


text:
🛑 CEOTR Home Page loads is down

In 223fa23, CEOTR Home Page loads (https://stg.ceotr.ca CEOTR Home) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CEOTR Home Page loads is back up in f75ded0 after 36 days, 12 hours, 34 minutes.


text:
You can CAS fire on some parts of the western caves of solaris

Description

Title. Happened in round # 837, it was one of the edges that allowed CAS.
Reproduction

Screenshots

Additional context

Going to assume this was fixed by the areas rework.

Should be good now,  you can run areas:loadmortar to check
The very edge do allow mortar fire since it's considered close enough to the cave entrance

authors: DrSmugleaf Vermidia


text:
odom frame not consistent at startup

ISSUE TEMPLATE ver. 0.4.0

Which TurtleBot3 platform do you use?

[ ] Burger

Which ROS is working with TurtleBot3?

[ ] ROS 2 Foxy Fitzroy

Which SBC(Single Board Computer) is working on TurtleBot3?

[ ] Raspberry Pi 4

Which OS you installed on SBC?

[ ] Ubuntu preinstalled server (20.04)

Which OS you installed on Remote PC?

[ ] Ubuntu 20.04 LTS (Focal Fossa)

Specify the software and firmware version(Can be found from Bringup messages)

Software version: [x.x.x]
Firmware version: [x.x.x]

Specify the commands or instructions to reproduce the issue.
on tb3_SBC: ros2 launch turtlebot3_bringup robot.launch.py
on remote computer: rqt

Copy and Paste the error messages on terminal.
rviz2: /tmp/binarydeb/ros-foxy-rviz-ogre-vendor-8.2.6/obj-x86_64-linux-gnu/ogre-v1.12.1-prefix/src/ogre-v1.12.1/OgreMain/include/OgreAxisAlignedBox.h:225: void Ogre::AxisAlignedBox::setExtents(const Vector3&, const Vector3&): Assertion `(min.x <= max.x && min.y <= max.y && min.z <= max.z) && "The minimum corner of the box must be less than or equal to maximum corner"' failed.
Aborted (core dumped)

Please describe the issue in detail.
When I launch robot.launch.py, everything launches fine with no error messages, sometimes. I am trying to understand why the odom frame starts at seemingly random positions at launch. Ideally, I want to start the odom frame to start at 0,0 or where the base_footprint starts for localization.

Sometimes one of the positions is so large I get the error message stated above and crashes the rviz2 gui.
please refer to the attached image and please let me know if I need to include more details or information.

Hi @huybeme.
Thank you for your questions.
I'm not sure what's wrong with your configuration, but I'm attaching an issue with a case similar to yours.
#594
Thank you.

Thank you for your response @ashekim, I looked over that ticket and am not sure that our issues are the same.
I also want to clarify that I am using an actual turtlebot3 burger and am not running simulation. Also both my computer and pi only have ROS foxy distribution installed.
Currently, I am only running rviz2 and repository:
https://github.com/ROBOTIS-GIT/turtlebot3/tree/foxy-devel

Hi @huybeme .
I'm sorry for the misunderstanding that the environment consists of Gazebo.
It appears that RVIZ crashes and throws an error if the robot position is outside of the grid.
In rviz2, Fixed Frame should be odom. And odom should be at the top of the TF Tree.

Could you please run the the command attached below on RemotePC??
$ ros2 launch turtlebot3_bringup rviz2.launch.py
I'd appreciate it if you let me know the results after you try.
Thank you

Hello @huybeme.
I want to know if you are using the slam package you developed instead of the slam package we provide.
Sorry for the trouble, but can you explain your running process?
Thank you.

@ashekim,
I am only launching robot bring up on my tb3. Prior to launching any other slam packages such as turtlebot3 cartographer, I have to check my position of my odom->base_footprint frame. Please refer to the photo containing translation of the frames below. The translation.y is very large and is what prevents rviz2 from opening and likely the cartographer as well.

In order to get rviz2 or cartographer to run/launch successfully, I must end the robot_bringup and re-launch until I get translation values that is within the grid.
Here is my rviz2 window with a robot bringup where translation is near 0.

Is this normal? Using teleoperation, I am able to move the robot around in rviz2 (and live) starting from that point. Also, I have tried to transform odom>base_footprint frame to set it to 0,0 without success. It seems like its original points takes precedent.
Let me know if there are any additional details that I can lay out.

@huybeme Hi, I have same problem using TB3 Burger and rviz2. The translation.y is often out of the grid. Sometimes, it will be small, like 2 or 10, but not 0. Did you solve this problem?

@xinxing-max Hello! I have not solved this issue definitively but as a bypass to getting things going, I sort of just keep re-launching the robot bringup through ROS until the random coordinates doesn't crash rviz. Please do keep me posted if you catch onto something that is causing this issue.

Hi,
I'm having the same problem. My tf between odom and base_footprint is:
transforms:
- header:
    stamp:
      sec: 1668100040
      nanosec: 707374661
    frame_id: odom
  child_frame_id: base_footprint
  transform:
    translation:
      x: 0.2770956275728272
      y: 8.311419928698475e+19
      z: 0.0
    rotation:
      x: 0.0
      y: 0.0
      z: 0.003235359852538116
      w: 0.999994766209616

As @xinxing-max said, the value of translation.y is huge!
Is there a way to "reset" or "recalibrate" the odometry, so it starts again at 0?
Thanks

hello @JLBicho , thanks for referencing this issue to #880. Were you successful in those steps? I won't have a chance to try it out for a few days but definitely will.

Hi @huybeme,
Yes, I was able to make it work correctly following those steps. It can happen more than once after each launch command.
If this doesn't work for your, please let us know.

Hi @JLBicho ,
Thanks for your suggestions. Your Edit2 did not always work for me. I could only restart turtlebot or relaunch bringup files to reset odom. Did those steps always work for you?

Hi @xinxing-max ,
For me it always works after stopping the bringup and launching again.
I'll try to burn again the foxy image to see if the problem disappears.
Thanks

This issue will be closed due to prolonged inactivity. Please try again with the Humble or Noetic versions, and if the issue persists, feel free to reopen it.

authors: JLBicho ashekim huybeme sunghowoo xinxing-max


text:
hipGetDeviceProperties gives non-existing name and wrong arch

I copied a hello world hip example in this repo https://gitlab.com/cscs-ci/ci-testing/ault-amdgpu/-/tree/40dc0bbda7036c0729e263f736b0d0a3dfdbbb52/ where I'm trying to run CI with slurm + container runtime using the rocm/dev-ubuntu-18.04 docker image. The host is running the ROCm 4.0.1 and I have 3 GPUs (1x Vega10, 2x Vega20).
I'm running 3 tasks/processes with respectively ROCR_VISIBLE_DEVICES=0, ROCR_VISIBLE_DEVICES=1, ROCR_VISIBLE_DEVICES=2 as set by slurm. It seems /opt/rocm/bin/rocm_agent_enumerator reports the right GPU info:
proc 0 has GPU   Uuid:                    GPU-3f50506172fc1a63                and visible ROCR_VISIBLE_DEVICES = 0
proc 1 has GPU   Uuid:                    GPU-3f4478c172fc1a63                and visible ROCR_VISIBLE_DEVICES = 1
proc 2 has GPU   Uuid:                    GPU-d15868e172fd5d69                and visible ROCR_VISIBLE_DEVICES = 2

However, the C++ code that calls hipGetDeviceProperties(&devProp, 0); produces 3x the same info:
 number of devices = 1
 agent prop name Device 66a1
 gcnArch 906
 gcnArchName gfx906

 number of devices = 1
 agent prop name Device 66a1
 gcnArch 906
 gcnArchName gfx906

 number of devices = 1
 agent prop name Device 66a1
 gcnArch 906
 gcnArchName gfx906

See more output here: https://gitlab.com/cscs-ci/ci-testing/ault-amdgpu/-/jobs/1015467883
The gcnArch is incorrect: should be 1x gfx900 and 2x gfx906 and the agent prop name is Device 66a1 which doesn't immediately make sense to me.
What's the problem here?

Ok, it might actually be a slurm issue... potentially it just selected the wrong node with 3 identical GPUs. Sorry for the confusion.

repo: ROCm-Developer-Tools/HIP


text:
gfx12 - correct float operator for ocp fp8, remove redudent self impl…

… and reduce build warning

gfx12
[----------] Global test environment tear-down
[==========] 22565 tests from 13 test suites ran. (240752 ms total)
[  PASSED  ] 22565 tests.
hipBLASLt version: 1100

repo: ROCm/hipBLASLt


text:
RPP Test Suite - Replace fstrings and subprocess calls to support old/new versions of Python

This PR adds relevant changes to support RPP test suite on old/new versions of python - tested on python 2.7, 3.5, 3.10

Replaces usage of fstrings with just strings for python backward compatibility
Replaces usage of subprocess.run with just subprocess.call / subprocess.Popen for python backward compatibility
Minor bugfix in test suite help display

@LakshmiKumar23 @kiritigowda This PR now passes CI and should support older and newer versions of Python.

@LakshmiKumar23 @kiritigowda This PR now passes CI and should support older and newer versions of Python.

I will update the ticket so QA can test and confirm

repo: ROCm/rpp


text:
Missing #include <limits>

When compiling MIOpen with OpenCL backend (release 4.3.0) on Arch Linux with GCC 11 I get several build errors like
MIOpen/src/include/miopen/float_equal.hpp:47:39: error: no member named 'numeric_limits' in namespace 'std'
               std::nextafter(x, std::numeric_limits<T>::lowest()) <= y

This is fixed by adding #include <limits> to float_equal.hpp:
diff --git a/src/include/miopen/float_equal.hpp b/src/include/miopen/float_equal.hpp
index e2441b60c..bd816cb22 100644
--- a/src/include/miopen/float_equal.hpp
+++ b/src/include/miopen/float_equal.hpp
@@ -28,6 +28,7 @@

 #include <algorithm>
 #include <cmath>
+#include <limits>
 #include <numeric>
 #ifdef _MSC_VER
 #include <iso646.h>

@tpkessler Thanks! It seems that GCC 11 started to explicitly require including <limits>, <memory>, <utility>, <thread>, according to gcc-11_header-dep-changes, it might have happened to clang too.
It might should have been fixed for most header files while this one is missed: have we only recently started testing "MIOpen with OpenCL backend (release 4.3.0) on Arch Linux with GCC 11"?

authors: junliume tpkessler

repo: ROCmSoftwarePlatform/MIOpen


text:
Preemptive PR on GPU'less compile

I am posting this PR preemptively so Tuna people can see my changes.
The idea is:

Set env var to compile only (for desired arch/num_cu) - through Precompile kernels and dump blobs into kern_db
Move kern cache to machine where the kernels will execute
Run through regular invokers code which will compile (if kernel not already present in kern_db) and execute. Note: the kernel will already be in kern_db from the compile step.

The following steps need to happen:
build with MIOPEN_SQLITE_KERN_CACHE=ON
To compile we need to set the following env vars:
export MIOPEN_DEBUG_CONV_GEMM=0  (GEMM code path has not been implemented for this)
export MIOPEN_CUSTOM_CACHE_DIR=test (this will be appended to ~/.cache)
export MIOPEN_FIND_ENFORCE=4
export MIOPEN_COMPILE_AND_RUN=0 (compile only, escape generic search early)
export MIOPEN_DEVICE_ARCH=gfx906 (or whatever other supported arch)
export MIOPEN_DEVICE_CU=60 (or any other num_cu)
Ro run:
export MIOPEN_DEBUG_CONV_GEMM=0 (this can also be left out and invokers will actually compile and run as well)
export MIOPEN_CUSTOM_CACHE_DIR=test (must be same location as compile so it can find compiled kernels)
export MIOPEN_FIND_ENFORCE=4
export MIOPEN_COMPILE_AND_RUN=2 (anything but 0, can also be left unset)
I have tested as such:

compile only(for arch/num_cu different than my own machine) and check kern_db to make sure its populated with compiled kernels.
moved ukdb to machine with arch/num_cu for which it was compiled for.
set appropriate env vars to pick up custom cache location and run.
executed above steps for both hip and opencl
tested with rocm 3.5

Feel free to propose some actual utest I can add.
Blocked by: https://github.com/ROCmSoftwarePlatform/MIOpen/pull/203

@alexandraBara This change affects +989 −1,791 lines. Are all of these changes really required?

@alexandraBara This change affects +989 −1,791 lines. Are all of these changes really required?

remember this branched off ddu/invoker-tune. most of the changes are his until he merges his branch the diff will be quite big.

@alexandraBara This change affects +989 −1,791 lines. Are all of these changes really required?

remember this is branched of ddu/invoker-tune. most of the changes are his until he merges his branch the diff will be quite big.

In this case could you please change base branch of this PR to ddu/invoker-tune? Please see https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/changing-the-base-branch-of-a-pull-request. This would greatly simplify reviewing.
I also recommend marking PR as a Draft to prevent it from merging into ddu/invoker-tune. Thanks.

@alexandraBara I've merged @DrizztDoUrden latest PR. Please update this PR with the latest develop and resolve conflicts.

@DrizztDoUrden Please review this PR.

@alexandraBara This change affects +989 −1,791 lines. Are all of these changes really required?

remember this is branched of ddu/invoker-tune. most of the changes are his until he merges his branch the diff will be quite big.

In this case could you please change base branch of this PR to ddu/invoker-tune? Please see https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/changing-the-base-branch-of-a-pull-request. This would greatly simplify reviewing.
I also recommend marking PR as a Draft to prevent it from merging into ddu/invoker-tune. Thanks.

Vasili's branch went in, so now its a PR from develop. Is this ok?

authors: alexandraBara atamazov daniellowell


text:
MFMA + LocalSplitU(LSU) support, plus small optimizations

added support for MFMA + LocalSplitU (only 2 or 4 can be used)
extended the range of GlobalSplitU to 4096 (to support larger DepthU with LSU)
extended the range of StaggerUStride to 2048 (to support larger DepthU with LSU)
improved the reject condition of DirectToVgpr + MIWaveGroup to cover LSU case
optimized TailLoop code

set 0 only for A or B instead of setting 0 both

optimized complex data type related code

removed redundant negation code for complex type
removed extra fixed vgpr allocation for alpha calculation. Use temporary vgpr instead.
unified CGEMM and ZGEMM code in MulMIoutAlphaToArch

optimized ShiftVectorComponents code to remove redundant mov for permute in case src and dst are the same vgpr
removed redundant shift code (0 bit shift)
optimized ComputeStoreVgprsMFMA to remove * 1 code when MIWG0=1
optimized cmp_lt + v_cndmask code to use v_min instead
added reject condition for LSU + DirectToLds
added reject condition for LSU + StoreCInUnroll
fixed typo
added test cases for MFMA + LSU in extended folder

This is a big PR, There's a few unrelated changes here that could have been separate PRs (better for reviewing, testing, debugging, error tracking). This will change some aspects of already tuned kernels. Have you tried running rocblas tests with these changes?

I have not tried rocblas yet.
These optimizations (except for MFMA+LSU) are basically removing redundant instructions.
From performance point of view, I do not expect it makes performance worse.
Additional optimizations are not much big. If necessary, I can create a separate PR for additional optimization part.

This is a big PR, There's a few unrelated changes here that could have been separate PRs (better for reviewing, testing, debugging, error tracking). This will change some aspects of already tuned kernels. Have you tried running rocblas tests with these changes?

I have not tried rocblas yet. These optimizations (except for MFMA+LSU) are basically removing redundant instructions. From performance point of view, I do not expect it makes performance worse.
Additional optimizations are not much big. If necessary, I can create a separate PR for additional optimization part.

Updated. I removed extra optimization.

The gfx90a extended failed last time (number 2).
I re-ran it and it passed this time (number 3).
I tried to reproduce the fail on my local node many times, but it all passed.
The fail is not reproducible.

repo: ROCmSoftwarePlatform/Tensile