text:
[SPARK-8189] [SQL] use Long for TimestampType in SQL

This PR change to use Long as internal type for TimestampType for efficiency, which means it will the precision below 100ns.

cc @rxin @JoshRosen

Merged build triggered.

Merged build started.

Test build #34539 has started for   PR 6733 at commit 309d2e1.

cc @liancheng to take a look at the column store stuff.

We also need to rewrite CastSuite to make it cover more stuff. The coverage currently is pretty low.

Test build #34539 has finished for   PR 6733 at commit 309d2e1.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Merged build triggered.

Merged build started.

Test build #34573 has started for   PR 6733 at commit 246ee74.

Test build #34573 has finished for   PR 6733 at commit 246ee74.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Merged build triggered.

Merged build started.

Test build #34577 has started for   PR 6733 at commit 8f77ee0.

Test build #34577 has finished for   PR 6733 at commit 8f77ee0.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

In-memory columnar part looks good. The test failure was because of timestamp precision change. We can just ignore those two Hive compatibility test cases.

Merged build triggered.

Merged build started.

Test build #34610 has started for   PR 6733 at commit 86fecfb.

Test build #895 has started for   PR 6733 at commit 86fecfb.

Test build #34610 timed out     for PR 6733 at commit 86fecfb     after a configured wait of 175m.

Merged build finished. Test FAILed.

Merged build triggered.

Merged build started.

Test build #34631 has started for   PR 6733 at commit 65cf2f1.

Merged build triggered.

Merged build started.

Test build #34632 has started for   PR 6733 at commit d9565fa.

Test build #895 timed out     for PR 6733 at commit 86fecfb     after a configured wait of 175m.

Test build #34632 has finished for   PR 6733 at commit d9565fa.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Jenkins, retest this please.

Merged build triggered.

Merged build started.

Test build #34638 has started for   PR 6733 at commit d9565fa.

Test build #34631 has finished for   PR 6733 at commit 65cf2f1.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

I'm going to merge this, but we should fix rest of the things later.
@JoshRosen would be great if you review this again too.

Test build #34638 has finished for   PR 6733 at commit d9565fa.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

authors: AmplabJenkins SparkQA davies liancheng rxin


text:
[SPARK-8770][SQL] Create BinaryOperator abstract class.

Our current BinaryExpression abstract class is not for generic binary expressions, i.e. it requires left/right children to have the same type. However, due to its name, contributors build new binary expressions that don't have that assumption (e.g. Sha) and still extend BinaryExpression.
This patch creates a new BinaryOperator abstract class, and update the analyzer o only apply type casting rule there. This patch also adds the notion of "prettyName" to expressions, which defines the user-facing name for the expression.

Merged build triggered.

Merged build started.

Test build #36326 has started for   PR 7174 at commit f31900d.

Test build #36326 has finished for   PR 7174 at commit f31900d.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
trait ExpectsInputTypes
trait AutoCastInputTypes
abstract class BinaryExpression extends Expression with trees.BinaryNode[Expression]
abstract class BinaryOperator extends BinaryExpression
abstract class BinaryArithmetic extends BinaryOperator
abstract class BinaryComparison extends BinaryOperator with Predicate

Merged build finished. Test PASSed.

source: gharchive/pull-request

authors: AmplabJenkins SparkQA rxin

repo: apache/spark

license_type: permissive

license_source: github-api


text:
STORM-3205 Optimization in TuplImpl

See Jira description for measurements and details

travis failure seems unrelated.

repo: apache/storm


text:
STORM-3307: Fixes error time on UI for time of last error on componen…

…t page

@govind-menon  Thank you for the patch.

authors: govind-menon kishorvpatil


text:
Incorrect Map of India

### some parts of india is not registered in indian territory i.e gilgit

@sumitrajKredx  Check if this is the one you are looking for.

source: gharchive/issue

authors: Yaswanth-Perumalla sumitrajKredx

repo: apache/superset


text:
refactor: QueryContext and QueryObject by decoupling and making it testable

A preparation task for #16991.
decoupling QueryContext and QueryObject classes from superset so they can be used in unit tests and future usages.
What have done:

separating the concerns: holding query parameters, objects creations (using factories), and query processing ( data retrieving) into separate modules and Classes.
move utils method to utils module
code cleanup

It relies on the current test cases since no logic was added
follows up:

query context and query context processor naming refactoring
keep refactoring query context processor: add factory, create processor for specific query context and not process them as procedure paradigm and remove the "ping-pong" calls with query_actions.
add caching parameters to query context
remove duplicates code

It relies on the current test cases since no logic was added

Note that this is super risky, because we don't have 100% unit test coverage.

It relies on the current test cases since no logic was added

Note that this is super risky because we don't have 100% unit test coverage.

I agree , but on the other hand if we want to move towards better coverage we need to start refactoring and do separation of concerns

I think if it's possible to split this change into smaller PRs, that will make it much more reviewable. Also, as the purpose of this PR is to make the code testable, I think it would be appropriate and necessary to add tests as part of the refactor.
And if there are any existing tests that cover parts of this, would you mind linking to them? As a reviewer I find tests very helpful for understanding the code.

authors: amitmiran137 betodealmeida ofekisr suddjian


text:
revert(#17570): fix dashboard editing for dashboards with non-ascii characters

SUMMARY
The bug: When a user saved a dashboard that contained a chart with non-ascii characters, the json_metadata became malformed, causing the dashboard to fail to load. The root cause is in #17570, which added positions into the json_metadata blob. Something went wrong in the json parsing which causes json_metadata to be truncated if the positions json contains emojis or other non-standard characters.
The root cause PR is pretty large, so we had trouble figuring out where this goes wrong. I suspect the fix forward is some combination of

Fix json parsing to handle non-ascii characters correctly
Validate that json_metadata is well-formed before writing it
Don't write the positions json, since it's already contained by a different column

Opening this revert PR as a starting point. This PR reverts #17570 and the accompanying #17707.
TESTING INSTRUCTIONS

to reproduce bug:

make dashboard
add chart with emoji in title
save dashboard
see error after save
refresh and see error on load

to verify this fix:

make dashboard
add chart with emoji in title
save dashboard
verify dashboard saved
refresh and view dashboard

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

@geido @etr2460 @graceguo-supercat @rusackas

@serenajiang I can't reproduce the issue with the steps that you shared on current master (07bbe84). Is there any other way to trigger the bug?
https://user-images.githubusercontent.com/15073128/146165307-da027133-b23e-4afd-b865-97775dfe3fc4.mov

Confirmed with @zhaoyongjie and @villebro that they can't repro either 🙁

@serenajiang @graceguo-supercat as hard as I tried I could not reproduce either. Do you have any additional information to help with reproducing the problem?
Please watch the video below. If we can reproduce it a fix might be a more desirable way to solve this problem and it might be as quick.
https://user-images.githubusercontent.com/60598000/146207191-82a7a9ee-8cb0-407d-abd6-2ca3ef8ea43e.mp4

for simple dashboard, once you edit dashboard and save it, you will see position_json column is inserted into json_metadata column, this is pretty easy to reproduce
for large dashboard:

set SUPERSET_DASHBOARD_POSITION_DATA_LIMIT = 16777215
create a large dashboard with charts and tabs, add emojis in tabs
make dashboard position_json > data is > 65k
edit dashboard, or save_as

Hello @graceguo-supercat Erik gave us a hint that the limit might be the problem not the non-ascii characters as you just correctly pointed out. I have released a PR that removes the positions from the metadata as they are unnecessarily saved there and tagged you there. Thank you 🙏

@geido @kgabryje  We initially thought the size was the problem too, but we realized that it was emojis from repro with small dashboard. More specific repro instructions:

create dashboard
add chart to dashboard and edit the chart title (in the dashboard) to contain an emoji, OR add a header with an emoji in it
save dashboard

Removing positions should fix this problem, since afaik, there aren't any other ways for emojis to get in. Thanks for the PR!

authors: geido graceguo-supercat kgabryje serenajiang


text:
feat(ssh_tunnel): APIs for SSH Tunnels

SUMMARY
Add our CREATE, UPDATE and DELETE APIs for SSH Tunnels.

[X] CREATE API

[X] API endpoint calling the command
[X] Tests

[X] UPDATE API

[X] API endpoint calling the command
[X] Tests

[X] DELETE API

[X] API endpoint calling the command
[X] Tests

[x] API + Schemas + Tests

[x] CREATE POST /api/v1/database
[x] UPDATE UPDATE /api/v1/database/
[x] DELETE DELETE /api/v1/database/{db_id}/ssh_tunnel

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

/testenv up

authors: Antonio-RiveroMartnez hughhhh


text:
TAJO-1260: Add ALTER TABLE ADD/DROP PARTITION statement to parser

I plan to implement ALTER TABLE ADD/DROP PARTITION statement stage by stage. For the first time, I'll add the statement to parser.

Syntax

ALTER TABLE table_name [IF NOT EXISTS] ADD PARTITION (key1 = 'val2', key2 = 'val2', ...) [LOCATION 'partition_path_of_directory'];
ALTER TABLE table_name [IF NOT EXISTS] DROP PARTITION (key1 = 'val2', key2 = 'val2', ...);

Hi @hyunsik
Thanks for your quick review. I used FileUtil.readTextFileFromResource instead of direct paths and I added more unit test cases.

Could you trigger the travisCI job?

Hi @hyunsik
I've just merged the master branch to TAJO-1260 branch.

+1
The patch looks good to me.
BTW, you need to consider inequality expression in DROP PARTITION. But, the current patch is good starting point. We can improve it later in another jira.
ALTER TABLE table_name [IF EXISTS] DROP PARTITION  (key1 [=|<|<=|>|>=|!=] 'val1', key2 ...,);

ALTER TABLE .. [ADD | DROP] PARTITION statement will cause error if this statement is given in JDBC driver or tsql. So, could you add some exception handling code to prevent ugly errors?

Hi @hyunsik
I added the exception handling code. If possible, could you check the pathch again?

Sure, I'll review it soon.

@blrunner Could you trigger TravisCI?

@hyunsik
Sure, I've just triggered it. :)

+1 ship it. Thanks.
It should be committed to only master branch.

authors: blrunner hyunsik

repo: apache/tajo


text:
Get ClientRespHeader failed in TSHttpTxnClose Hook

When I get ClientRespHeader statistics for response code at TSHttpTxnClose hook, but i find TSHttpTxnClientRespGet return false。
ats version: 7.1.8
g_txn_close_contp = TSContCreate(MetricsTxnCloseHookHandle, nullptr);
TSHttpHookAdd(TS_HTTP_TXN_CLOSE_HOOK, g_txn_close_contp);

static int
MetricsTxnCloseHookHandle(TSCont /* contp ATS_UNUSED */, TSEvent event, void *edata)
{
  .....
  status = HttpStatusGet(txnp);
  .....
}

static TSHttpStatus
HttpStatusGet(TSHttpTxn txnp)
{
  TSHttpStatus status;
  TSMBuffer req_bufp;
  TSMLoc req_loc;
  TSAssert(TSHttpTxnClientRespGet(txnp, &req_bufp, &req_loc) == TS_SUCCESS);
  status = TSHttpHdrStatusGet(req_bufp, req_loc);
  TSHandleMLocRelease(req_bufp, TS_NULL_MLOC, req_loc);
  return status;
}

Carsh Stack:
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007f1ea232b51a in __GI_abort () at abort.c:118
#2  0x00007f1ea49a413b in ink_abort (message_format=message_format@entry=0x7f1ea49b0927 "%s:%d: failed assertion `%s`")
    at ink_error.cc:99
#3  0x00007f1ea49a2465 in _ink_assert (
    expression=expression@entry=0x55f4fab5e848 "sdk_sanity_check_mbuffer(bufp) == TS_SUCCESS",
    file=file@entry=0x55f4fab5e580 "InkAPI.cc", line=line@entry=3913) at ink_assert.cc:37
#4  0x000055f4fa845a31 in _TSReleaseAssert (line=3913, file=0x55f4fab5e580 "InkAPI.cc",
    text=0x55f4fab5e848 "sdk_sanity_check_mbuffer(bufp) == TS_SUCCESS") at InkAPI.cc:396
#5  TSHttpHdrStatusGet (bufp=<optimized out>, obj=<optimized out>) at InkAPI.cc:3913
#6  0x00007f1e943ef646 in HttpStatusGet (txnp=0x7f1b06589000) at src/ats_metrics.cc:31
#7  MetricsTxnCloseHookHandle (event=<optimized out>, edata=0x7f1b06589000) at src/ats_metrics.cc:186
#8  0x000055f4fa83ec12 in INKContInternal::handle_event (this=0x7f1ea0bfba80, event=60012, edata=0x7f1b06589000)
    at InkAPI.cc:1044
#9  0x000055f4fa91c59e in HttpSM::state_api_callout (this=this@entry=0x7f1b06589000, event=event@entry=60000,
    data=data@entry=0x0) at HttpSM.cc:1448
#10 0x000055f4fa9264e2 in HttpSM::state_api_callback (this=this@entry=0x7f1b06589000, event=event@entry=60000,
    data=data@entry=0x0) at HttpSM.cc:1321
#11 0x000055f4fa854798 in TSHttpTxnReenable (txnp=0x7f1b06589000, event=TS_EVENT_HTTP_CONTINUE) at InkAPI.cc:5894
#12 0x00007f1e943ede77 in PluginInitHandleHook (event=<optimized out>, edata=0x7f1b06589000) at src/plugin_init.cc:88
#13 0x000055f4fa83ec12 in INKContInternal::handle_event (this=0x7f1ea0bfb8a0, event=60012, edata=0x7f1b06589000)
    at InkAPI.cc:1044
#14 0x000055f4fa91c59e in HttpSM::state_api_callout (this=0x7f1b06589000, event=<optimized out>, data=<optimized out>)
    at HttpSM.cc:1448
#15 0x000055f4fa925634 in HttpSM::kill_this (this=0x7f1b06589000) at HttpSM.cc:6926
#16 0x000055f4fa92630c in HttpSM::attach_client_session (this=<optimized out>,
    client_vc=client_vc@entry=0x7f15d886c1e8, buffer_reader=<optimized out>) at HttpSM.cc:620
#17 0x000055f4fa87d51a in ProxyClientTransaction::new_transaction (this=this@entry=0x7f15d886c1e8)
    at ProxyClientTransaction.cc:60
#18 0x000055f4fa901960 in Http1ClientSession::new_transaction (this=0x7f15d886bf00) at Http1ClientSession.cc:507
#19 Http1ClientSession::release (trans=0x7f15d886c1e8, this=0x7f15d886bf00) at Http1ClientSession.cc:471
#20 Http1ClientSession::start (this=0x7f15d886bf00) at Http1ClientSession.h:65
#21 0x000055f4fa87c8be in ProxyClientSession::handle_api_return (this=0x7f15d886bf00, event=<optimized out>)
    at ProxyClientSession.cc:204
#22 0x000055f4fa87c96f in ProxyClientSession::state_api_callout (this=0x7f15d886bf00, event=<optimized out>,
    data=<optimized out>) at ProxyClientSession.cc:160
#23 0x000055f4fa8544b3 in Continuation::handleEvent (data=0x0, event=60000, this=0x7f15d886bf00)
    at /data/bdcdn_open_soft/soft/source/trafficserver-7.1.8/iocore/eventsystem/I_Continuation.h:152
#24 TSHttpSsnReenable (ssnp=0x7f15d886bf00, event=TS_EVENT_HTTP_CONTINUE) at InkAPI.cc:4603
---Type <return> to continue, or q <return> to quit---
#25 0x00007f1e943ee3e7 in MetricsSsnStartHookHandle (event=<optimized out>, edata=0x7f15d886bf00)
    at src/ats_metrics.cc:41
#26 0x000055f4fa83ec12 in INKContInternal::handle_event (this=0x7f1ea0bfb900, event=60013, edata=0x7f15d886bf00)
    at InkAPI.cc:1044
#27 0x000055f4fa87ca2f in ProxyClientSession::state_api_callout (this=this@entry=0x7f15d886bf00, event=event@entry=0,
    data=data@entry=0x0) at ProxyClientSession.cc:146
#28 0x000055f4fa87ce1b in ProxyClientSession::do_api_callout (this=this@entry=0x7f15d886bf00,
    id=id@entry=TS_HTTP_SSN_START_HOOK) at ProxyClientSession.cc:182
#29 0x000055f4fa8fd9de in Http1ClientSession::new_connection (this=0x7f15d886bf00, new_vc=<optimized out>,
    iobuf=<optimized out>, reader=<optimized out>, backdoor=<optimized out>) at Http1ClientSession.cc:219
#30 0x000055f4fa8f7da4 in HttpSessionAccept::accept (this=0x7f1e9738c000, netvc=0x7f04b9e69b00, iobuf=0x7f1e9685e700,
    reader=0x7f1e9685e718) at HttpSessionAccept.cc:72
#31 0x000055f4fa87c230 in ProtocolProbeTrampoline::ioCompletionEvent (this=0x7f102ea7a000, event=<optimized out>,
    edata=<optimized out>) at ProtocolProbeSessionAccept.cc:107
#32 0x000055f4fab1cb8a in Continuation::handleEvent (data=0x7f04b9e69c30, event=100, this=<optimized out>)
    at /data/bdcdn_open_soft/soft/source/trafficserver-7.1.8/iocore/eventsystem/I_Continuation.h:152
#33 read_signal_and_update (vc=0x7f04b9e69b00, event=100) at UnixNetVConnection.cc:144
#34 read_from_net (nh=<optimized out>, vc=0x7f04b9e69b00, thread=<optimized out>) at UnixNetVConnection.cc:384
#35 0x000055f4fab0dbee in NetHandler::waitForActivity (this=0x7f1e9f0c7c00, timeout=<optimized out>) at UnixNet.cc:497
#36 0x000055f4fab58112 in EThread::execute_regular (this=0x7f1e9f0c3d40) at UnixEThread.cc:248
#37 0x000055f4fab56519 in spawn_thread_internal (a=0x7f1ea1f482f0) at Thread.cc:84
#38 0x00007f1ea313e4a4 in start_thread (arg=0x7f1e9d706700) at pthread_create.c:456
#39 0x00007f1ea23dfd0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

I'm not sure why this problem，  even if the request and response are problematic, ats should respond correctly.

Did this happen in an optimized or a debug build?  In an optimized build, TSAssert() is optimized out, so TSHttpTxnClientRespGet may have returned TS_ERROR.  I don't think ClientRestGet will always return TS_SUCCESS in the close hook unless you are adding the continuation in the SEND_RESPONSE_HDR hook.   Have you seen this problem in ATS9?

@bryancall can i work on this?
One possible cause of this error is that the client response header has already been released or is not available for some reason. To address this issue, you can add a check to ensure that the client response header is available before trying to retrieve it.
Another possible cause of this error is that the TSHttpTxnClose hook is being called after the client response has already been closed. In this case, you may need to move the hook to an earlier point in the transaction flow where the client response header is still available.

authors: Pheewww ywkaras zhangqiang-01

repo: apache/trafficserver


text:
Display auto_option hint in all cases

This takes the HINT added by @dragon512 a step further. The cases are as follows:
- feature enabled, option enabled: hints how to disable
- feature enabled, option disabled: no hint, something broke
- feature disabled, option enabled (AUTO): hints how to require
- feature disabled, option disabled: hints how to enable
I made a distinction between enabling and requiring. If the option is set to AUTO, it is already enabled, in some sense. If the user sets an option to ON, the user is requiring that the corresponding feature must be turned on, and should expect an error if a dependency is missing.

[approve ci clang-format]

[approve ci format]

authors: JosiahWI bneradt


text:
Coverity: 1381603 1381604 fix to traffic_layout on engine and file_system

A coverity fix to the errors reported in the runroot functionality in traffic_layout.
Modification in engine.cc and file_system.cc.
And a small README document fix.

[approve ci]

[approve ci autest]


text:
Fix potential crash from returning string_view.

Due to poor design of std::string_view, it does not handle nullptr correctly in its constructor.

Does the fix for this need to go to 7.1.2?

[approve ci autest]

authors: SolidWallOfCode dragon512 zwoop


text:
detect and fix stms log field having epoch time ms for intercept plugins.

For intercept plugins like stats_over_http and slice the stms logging field may end up being set to epoch time in ms.
This fix detects that condition inside HttpSM by only allowing TS_MILESTONE_SERVER_CLOSE to be set if
TS_MILESTONE_SERVER_CONNECT was already set.
LogAccess::marshal_server_resp_time_ms uses this code to calculate this stms field.
m_http_sm->milestones.difference_msec(TS_MILESTONE_SERVER_CONNECT, TS_MILESTONE_SERVER_CLOSE)

This is a mitigation in the code.
resolves #7196

Hey @traeak, we observed this occurring in our environment due to requests handled by an intercept plugin. I reviewed the fix provided here in the context of running an intercept plugin, and I believe that it will resolve the issue. It appears that there's only one place where the TS_MILESTONE_SERVER_CONNECT milestone timing is set, inside HttpSM::do_server_open(bool), and if that method is not called, the subsequent logging calculations that leverage this milestone produce unexpected results as you mentioned above.
I believe we should merge this PR. 👍

Cherry-picked to v9.0.x branch.

Cherry-picked to v9.0.x branch.

Cherry-picked to 8.1.x

Cherry-picked to 8.1.x

authors: elsloo traeak zwoop


text:
Test: add nameserver to regex_remap test.

This test uses "localhost" and therefore should have a local nameserver.

Looks like a bad cherry pick.

Fixed.

[approve ci autest]

Gah. How can this fail -
Running Test traffic_dump_sni_filter:.. Failed

authors: SolidWallOfCode randall


text:
TVM v0.9.0.rc0 Release Candidate Notes

Please leave any comments or edit this issue directly to adjust the release notes! Also see the rc0 vote thread in #12103.
Introduction
The TVM community has worked since the v0.8 release to deliver many exciting features and improvements. v0.9.0 is the first release on the new quarterly release schedule and includes many highlights, such as:

MetaSchedule's full implementation
ARM cascading scheduler for Arm Ethos(TM)-U NPUs
Collage which brings tuning to BYOC
Several microTVM improvements
New tvm.relay.build parameters: runtime=, executor=,
AOT: support for the C++ runtime (with llvm and c targets only) and support for host-driven AOT in the C runtime
Hexagon RPC support

Testing via Hexagon SDK simulator and on device via Snapdragon-based HDK boards and phones
AOT and USMP support
Threading
Initial op support

MLF: support for multiple modules in a single MLF artifact
Several TIR schedule primitives and transforms including (abridged):

schedule.transform_layout - Applies a layout transformation to a buffer as specified by an IndexMap.
schedule.transform_block_layout - Applies a schedule transformation to a block as specified by an IndexMap.
schedule.set_axis_separators - Sets axis separators in a buffer to lower to multi-dimensional memory (e.g. texture memory).
transform.InjectSoftwarePipeline - Transforms annotated loop nest into a pipeline prologue, body and epilogue where producers and consumers are overlapped.
transform.CommonSubexprElimTIR - Implements common-subexpression elimination for TIR.
transform.InjectPTXAsyncCopy - Rewrites global to shared memory copies in CUDA with async copy when annotated tir::attr::async_scope.
transform.LowerCrossThreadReduction - Enables support for reductions across threads on GPUs.

And many more! See the list of RFCs and PRs included in v0.9.0 for a complete list, as well as the full change list.

RFCs
These RFCs have been merged in apache/tvm-rfcs since the last release.

[RFC] TUNIP: TVMScript Unified Printer (#74) (48d47c5)
[RFC][Backend] RFC-CSI-NN2-Integration (#75) (cfcf114)
[RFC] Introducing DeclBuffer (#70) (87ff1fa)
[RFC][MLF] Model Library Format with Multiple Modules (#76) (f47c6ad)
[RFC] UMA Universal Modular Accelerator Interface (#60) (6990e13)
[RFC] DietCode: An Auto-Scheduler for Dynamic Tensor Programs (#72) (a518000)
[RFC] Quarterly Releases (#67) (70293c7)
RFC-BYOC-DNNL-Integration (#73) (7aed0ca)
[RFC] Relay Next Roadmap (#69) (ac15f2a)
RFC: clarifying buffer declaration and access (#63) (de4fe97)
Inclusive Language RFC (#68) (#68) (4203bd2)
[USMP] Adding U4 usecase (#65) (b9e246f)
Collage RFC (#62) (23250f5)
Replace codeowners with more relevant automation (#58) (540c1f8)
[RFC][TIR] Layout transformations on buffer access (#39) (b675ef8)
Module Based Model Runtime for AOT (#46) (d9dd6eb)
@slow test RFC (#55) (9b6203a)
[RFC][Roadmap] TVM Continuous Integration & Testing Roadmap (#54) (41e5ba0)
Bring PackedFunc into TVM Object System (#51) (2e0de6c)
[RFC][OpenCLML] OpenCLML integration as BYOC (#52) (f5ef65f)
Introduce the Arm(R) Ethos(TM)-U Cascading Scheduler (#37) (f9fa824)
[RFC][Roadmap] microTVM roadmap (#53) (1b14456)
Add Managed Jenkins Infrastructure for TVM RFC (#49) (a3a7d2c)
TVM Roadmap RFC (#50) (263335f)
[RFC] Integrate LIBXSMM with TVM. (#47) (1a3d4f1)
[RELAY][AST] Add virtual device as a first class field to Relay expressions (#45) (67c39d2)

What's Changed
Note that this list is not comprehensive of all PRs and discussions since v0.8. Please visit the full listing of commits for a complete view: https://github.com/apache/tvm/compare/v0.8.0...v0.9.0.rc0.
AOT

#11208 - Calculate used memory at the callsite of primitive functions
#11365 - Fix function number datatype from char to uint16_t
#11091 - Enable A-Normal Form in the AOT executor
#10753 - Support LLVM backend with C++ runtime
#10518 - Use python temporary directory for AOT tests
#10337 - BugFix of workspace calculation
#10282 - [runtime] Add Metadata classes for AOTExecutor
#9501 - [3/3][DeviceAPI] Wire up cpacked Device API context
#9500 - [2/3][DeviceAPI] Add Hooks for Activate/Deactivate/Open/Close
#9395 - [1/3][DeviceAPI] Connecting devices structure to relevant operators

BYOC

#11474 - Two helper passes for external codegen using RelayToTIR custom pass machinery
#11144 - Remove support for run-time linked-params from codegen
#10590 - Add order to functions in C Codegen
#11638 - [DNNL][CBLAS]Unifles all MKLDNN/DNNL  to DNNL
#11619 - RelayToTIR custom codegen passes can still depend on dynamic shape functions
DNNL - #11902, #11642, #11513, #11571, #11560, #11345, #11111, #10837, #10421, #9995, #9797
TensorRT - #11923, #11203, #10759, #10772, #10388
CMSIS-NN - #11732, #11625, #10939, #11013, #10817, #10563, #10224, #10148, #10100, #9338, #9531, #9409, #9331
OpenCLML - #10243
CUTLASS - #11631, #10185, #10177, #10110, #10036, #9899, #9820, #9800, #9795, #9746, #9737, #9698, #9595, #9571
CUDNN - #10997, #9986, #9948
ACL - #10801
PTX - #10855, #10339, #9909
CUBLAS - #10826, #10820

CI

#11313 - Refactor of tvm.testing.requires_* annotations
#11666 - Enable pylint for tests/python/ci
#11657 - Apply linting rules to AOT tests
#11380 - Restructure Jenkinsfile
Automation - #11813, #11775, #11480, #11437, #10833, #10056, #9973, #9934
User experience improvements - #11470, #11329, #11553, #11497, #11051, #10933, #10960, #10525, #10425, #10322, #10121, #9971, #9554, #9752, #9556
Reduce CI runtime - #11402, #11349, #11258, #11132, #10946, #10743, #10359
Code cleanups - #10968, #10740

Frontends

PaddlePaddle - #11537, #9724, #9564
TFLite - #10915, #10566
Oneflow - #11321, #11036, #8790
PyTorch - #11190, #10504, #10184, #10091
ONNX - #10949, #9438, #9186, #9493, #9475
Keras - #7006

Hexagon

#11549 - Initial clip operator for Hexagon
#11834 - Add op resize2d for hexagon
#11559 - Softmax slice op initial version
#11529 - Slice ops added - add, subtract, multiply
#11720 - [testing] add max_pool2d benchmark
#11417 - Implement avg_pool2d slice op
#11653 - Add HexagonThreadManager
#11547 - Run single RPC server on Android in each testing session
#11490 - [testing] add TVMScript elemwise-add
#11400 - [testing] refactor benchmark-table code
#11277 - moves conftest.py to tvm.contrib.hexagon so outside repos can access the testing fixtures
#11319 - Add unit tests for Hexagon Device API
#11279 - Add USMP tests
#11283 - Update Readme
#11239 - capture gtest output and return over FFI
#11175 - Add schedule and test for conv2d_transpose_nchw
#11018 - [Runtime] Add QuRT thread pool backend
#11145 - Add support for on-device unit testing using gtest
#11138 - Add test for depthwise conv2d schedule
#11016 - Add test for registered schedules
#11104 - Add mobilenet test
#11090 - Delete offload runtime, move files to right places
#11065 - AoT with LLVM Codegen on Hexagon
#11025 - Deprecate USE_HEXAGON_DEVICE, introduce USE_HEXAGON
#10604 - HVX scheduling and bench-marking of TE element-wise add
#10905 - [LLVM] Enable/test tensorized Hexagon DMA on 2d transformed layout
#10907 - Move aot/graph_executor interactions into launcher
#10919 - Register basic strategies and schedules for common operators
#10904 - Add unit tests executing 2-d VTCM usage
#10910 - Refactor to keep HexagonBuffer private to the device api
#10908 - [LLVM][CodeGen] Make CodeGenHexagon a subclass of CodeGenCPU
#10878 - Generalized HexagonBuffer::CopyTo/CopyFrom
#10846 - Support both 1-d and 2-d VTCM allocations
#10581 - Improved ergonomics of HexagonLauncher in unit tests.
#10616 - Refactor tvm.contrib.hexagon, NFC
#10612 - Deprecate SDK 3.x, rewrite HexagonSDK.cmake
#10586 - Codegen for 2d Load/Store
#10558 - Generalize builtin for Nd memory alloc with storage scope and add lowering for VTCM / Hexagon
#10543 - [Runtime][PipelineExecutor] Add the pipeline internal forwarding logic.
#10507 - Add doc on TVM - Hexagon RPC flow
#10520 - Resolve breakage in test_hexagon/test_cache_read_write
#10311 - [runtime]AOTExecutor implementation for C Codegen
#10454 - Allow execution on target or simulator from HexagonLauncher
#10365 - Lower cache_read and cache_write to Hexagon DMA via tensorize
#10361 - RPC server/client for simulator
#10302 - [CI]Add Hexagon Tests to pipeline
#10263 - [Docker]Add docker file and scripts
#10227 - Refactor Hexagon.cmake
#10217 - Adding support for Hexagon User DMA Engine
#10068 - Update hexagon API build instruction and cleanup hexagon_proxy_rpc
#9970 - Do not auto-build apps when building TVM
#9736 - Add unit tests for HexagonBuffer
#9525 - Add Hexagon VTCM and discontiguous allocation support
#9631 - Add RPC Mechanism for Hexagon
#9473 - cleanup Hexagon conv2d tests

MetaSchedule

#11884 - Postproc: Rewrite-Layout
#11848 - [OpStrategy] Support MetaSchedule Layout
#11845 - [Relay][Pass] Meta-Schedule-Layout-Rewrite
#11758 - [Runtime] Enhance Runner RandomFill
#11683 - Distributed Measurement
#11751 - [Minor] Organize Testing Scripts
#11735 - Modify Profiler Timers
#11727 - Developer Ergonomics Enhancement II
#11692 - Apply-History-Best Task Filtering
#11486 - Add Profiler Support For Tuning Efficiency Optimization
#11680 - JSONDatabase Utilities
#11641 - Generate MetaSchedule Dataset
#11622 - Developer Ergonomics Enhancement
#11604 - Resolve dependencies between header files
#11587 - Add Testing Script with ONNX Support
#11590 - Evo Independence from TaskScheduler
#11534 - No explicit unrolling for spatial PrimFunc
#11512 - Enable Task Filtering
#11177 - AutoBind rule and MutateThreadBinding
#11157 - Logging Interface Unification
#11088 - Auto tensorization for CPU / GPU dot product
#10986 - [Refactor] Introduce TuneConfig
#11020 - [Metaschedule, Refactor] Move MultiLevelTilingNode decl to a header
#10927 - [Refactor] Clarify Integration Logic
#10876 - Add utility API to ease using manual schedules
#10885 - [BugFix] Fix skipped tests
#10366 - Add Gradient Based Task Scheduler
#10823 - Fine-Grained Rewrite Unbound Block
#10793 - Add demonstration of selectively tuning relay ops with TIR schedules
#10811 - Support grouping in the cost model
#10810 - Extract task weights during task extraction
#10782 - [TIR]Estimate TIR FLOPs
#10776 - Misc updates for tuning end-to-end workloads
#10689 - Upstream the leftover changes
#10648 - [Meta Schedule] Refactor meta schedule testing utils
#10578 - New relay backend for meta schedule task extraction
#10534 - Bug Fix for Relay Integration
#10501 - Update scripts for subgraph tuning
#10497 - Refactor testing workloads
#10461 - Enable AutoTVM-style template-based search space
#10368 - Fix Cyclic Dependency in PyClass Family
#10403 - Arithmetic analysis
#10367 - Update Tuning Interfaces.
#10079 - [M4a] User-API: Tune-TE/TIR/Relay
#10081 - [M4a] Rewrite-Cooperative-Fetch
#10055 - [M4b] Testcases for TensorRT builder/runner
#10092 - [M4a] Mutator: Mutate-Tile-Size
#10096 - [M4a] Mutator: Mutate Parallel
#10071 - [M4a] PostProcessor: Rewrite-Parallel-Vectorize-Unroll
#10043 - [M4a] Schedule Rule: Multi-Level-Tiling
#10045 - Mutator: Mutate-Unroll
#10033 - [M4a] Schedule Rule: Parallelize-Vectorize-Unroll
#10027 - [M4a] PostProcessor: Rewrite-Unbound-Block
#10028 - Mutator: Mutate-Compute-Location
#9997 - [M4a] PostProcessor: Disallow-Dynamic-Loop
#9994 - [M4a] Schedule Rule: Cross-Thread-Reduction
#10013 - [M4a] PostProcessor: Rewrite Reduction Block
#9975 - [M4a] Schedule Rule: Add-RFactor
#9945 - [M4a] PostProcessor: Verify-GPU-Code
#9940 - [M4a] Schedule Rule: Random-Compute-Location
#9943 - [M4a] Schedule Rule: Auto-Inline
#9860 - [M3c] Add Per-Store-Feature
#9859 - [M3c] XGB-based Cost Model
#9836 - [M4a] Add EvolutionarySearch Search Strategy
#9799 - [M4a] Add ReplayFunc Search Strategy
#9789 - [M3c] Update TuneContext, TaskScheduler & Search Strategy Design
#9780 - [M3c] Add More Measure Callbacks
#9761 - [M4a] Add ScheduleRule class & PostOrderApply space generator
#9760 - [M3c] Random Feature Extractor

MicroTVM

#11741 - Refactor RVM scripts and fix DNS network issue
#11472 - [ARM]Add tests for arm schedules
#11634 - Update pyproject to python3.7
Zephyr support - #11650
RPC - #11227, #10967

Relay

#11825 - [realy][pass]add split infer shape with convert op layout pass
#11674 - Finish implementations of WithFields
#11481 - IndexedGraph improvements in preparation for Collage
#11432 - Plumb external codegen target via Target.current()
#11494 - [Pass] Add MaxPool, AvgPool to FoldExplicitPadding
#11183 - Add unidirectional sequence lstm
#11442 - Add 'static_library' runtime::Module
#11413 - [Topi]Support for FP16 ERF on CPU.
#11382 - Finish support for list-of-targets
#11386 - [Tests] Replace the Relay interpreter with the VM in the op tests
#11224 - Support i16, f16 scalars in Relay text
#11337 - Fix eltwise alter op layout for broadcast axis
#11199 - Flexible shape dispatch transformation
#11173 - Support 'external codegen targets'.
#10996 - Add FlattenAtrousConv transformation
#10871 - [CUDNN] Add cuDNN as a Relay partitioning target (BYOC)
#10787 - [Pass][Bugfix] Disable re-use of non-flat buffers in StorageRewrite.
#10378 - [FQ2I] Add leaky relu to FQ21
#10400 - RelayViz graphviz renderer
#10352 - [VIRTUALDEVICE] Change syntax for device planning and store parameter virtual devices in virtual_device_ field
#10310 - [ARM_CPU] Conv2d int8 intrinsic for cortex-A72
#10085 - RelayViz interface and terminal ast-dump
#10239 - Add a conversion of individual operations in FQ2I pass.
#10236 - [Refactor] Clean up type relations that are declared as template for no reason
#10156 - Fix broadcast InferCorrectLayout
#10026 - [VM] Relay VM memory liveness/lifetime analysis
#10089 - [Pass] Add a relay pass to extract fake quantized ops
#9690 - Change function constructors to WithFields
#10069 - [DefuseOps pass] bug fix: To support function body types other…
#9954 - Add conv2d_backward_weight op (without topi)
#9838 - [FoldScaleAxis] Support dense and bias_add op in fold scale axis
#9816 - Add sliding_window operator
#9874 - Add a JSON converter for 0.7 -> 0.8 and 0.8 -> 0.9
#9735 - [AMP][Pass][Typing] Add faster type inference
#9723 - [Frontend] Add Span filling for frontends to Relay
#9749 - Fix invalid shape function for "copy" operator
#9759 - s/SEScope/VirtualDevice/g
#9734 - Support large constants saved/loaded outside of VM executable
#9613 - Re-run PlanDevices after LowerTE to flow new memory scope constraints.
#9693 - PlanDevices supports 'free' on_device annotations
#9641 - [AST] Add virtual_device as a first class field in Relay
#9483 - Switch the VM to use the LowerTE pass instead of TECompiler::{Lower,LowerShapeFunc}.
#9569 - WithFields method for Call, Function, Var, TupleGetItem, If, Let, RefCreate, RefRead, RefWrite, Match, and Clause
#9533 - WithFields for Tuples
#9550 - Prepare for switching VM to LowerTEPass.
#9542 - Prepare DeadCodeElimination for running post LowerTEPass/ManifestAlloc.
#9352 - [TVMC]Introduce executor and runtime parameters
#9457 - Add the Arm(R) Ethos(TM)-U NPU identity operator
#9326 - Switch PlanDevices pass to be w.r.t. SEScopes instead of DLDeviceTypes.
QNN - #11228, #10718, #10086, #10053, #9637, #9982

Runtime

#11334 - [PipelineExecutor] Add graph manually splitting logic into the unit test.
#11133 - [PipelineExecutor] Refactor PipelineExecutor.py and Add cross compile support for pipeline executor.
#11172 - Move WrapTimeEvaluator from RPC to profiling, NFC
#10990 - [PipelineExecutor]Add forwarding queue logic for set input.
#10953 - [Vulkan] Add RGP support to TVM for vulkan device
#10723 - [PipelineExecutor] Getting the asynchronous output
#10283 - AOTExecutor implementation and c target code-generator
#9802 - [ThreadPool]Refactor affinity function and support CPU affinity list setting.
#10234 - [Pipeline Executor] multiple threads management and the data forwarding notification mechanism.
#10326 - Improved log information with function signature
#10032 - [PackedFunc] Bring PackedFunc into TVM Object System
#10082 - [PipelineExecutor] Pipeline Executor Sequential execution
#10010 - [PipelineExecutor] Add Pipeline Executor Interface
#9846 - [Pipeline executor] Global parameters group name and runtime modules parameters map.
#9889 - [GraphExecutor] Add API get_input_info to graph_executor
#9751 - [Pipeline Executor] Add the map logic of global input and subgraph input.

TE

#11589 - Support schedulable TIR compute definitions in TOPI
#11341 - Optimized version of concatenation layer
#10561 - [TECompiler] Decouple TE compute and schedule lowering in ScheduleBuilder

TIR

#11592 - HoistExpression, generalization of HoistIfThenElse
#11870 - [Pass] Remove-Weight-Layout-Rewrite-Block
#11740 - [TIR, analysis] Add GetAutoTensorizeMappingInfo to generate transforms for auto tensorization
#11585 - Add preserve-unit-iters
#11677 - Register CUDA WMMA tensor intrinsics
#11658 - [TIR, CUDA] Add pass to replace global to shared memory copy with cp.async
#11624 - [Schedule] Allow named block and buffer arguments in Schedule
#11628 - [PASS] Refactor a couple of TIR passes - BindTarget, AnnotateEntryFunc, Filter, LowerInitBlock
#11574 - CSE pass : Restrict the equivalence to be decided by a normal form - avoids comparison of terms
#11575 - Schedule Primitive: Add-Unit-Loop
#11515 - Add schedule primitive ReIndex
#11524 - [Arith] Additional Simplifications Inside Conditionals
#11485 - Add schedule primitive TransformBlockLayout
#11495 - [Software pipeline] Fix hardcoded index in access_ptr rewriting, add a GPU test with depth 4
#11269 - [Schedule] Transform layout quality of life
#11355 - Support tensorization using ldmatrix + MMA
#11289 - [Schedule] Allowed typing.Tuple in tir.schedule._type_checker
#11317 - Support affine expressions as indices in reverse compute inline
#11235 - [Arith] Implemented padded inverses in IndexMap
#11238 - [ROOFLINE] Calculate roofline from existing TIR PrimFunc
#11225 - Add schedule primitive SetAxisSeparator
#11110 - Get read/write access precisely for opaque access.
#11106 - Enhance software pipeline validation and fix predicate of epilogue
#10843 - StmtFunctor RenewDefs
#11075 - Add function to tile a block according to a given tensor intrinsic
#11050 - Utility function to decide loop mapping for auto tensorization
#11009 - [ROCM] DP4A intrinsic support for TE/TIR
#10925 - VNNI and ARM dot product intrinsic for tensorization
#10887 - [Schedule] Relax reorder primitive's affine binding check
#10732 - [Analysis] Add SuggestIndexMap for layout rewriting
#10538 - [Schedule] Transform layout
#10638 - Change the behavior of read/write region analysis for reduction blocks.
#10705 - Use local complete block and local reduction block to identify compact dataflow
#10671 - Tuple Reduction Support in CreatePrimFunc
#9727 - [TE]Implement layout transformations, non-flat memory buffers
#10405 - [TensorIR] Update VerifyGPU
#10401 - [TensorIR] Renormalize split pattern
#10112 - [TIR, Relay] improve bfloat16 support
#8509 - Tir constants integration into compilation pipeline
#9996 - add support for multi-blocking layout and their transformation
#10066 - Add software pipelining
#10207 - Support sub warp reduction for CUDA target.
#9482 - Implementation of Common Subexpression Elimination for TIR
#9527 - Allow compute_at create block predicate for non-trivial bounds and support floordiv pattern
#10158 - [Schedule] Update compact_dataflow constraint
#9871 - [Schedule] Blockize and Tensorize
#10016 - [BugFix]Fix cross-thread reduction when single reduction loop with predicate
#9880 - Encode conditional accesses info into block read/write regions
#9699 - Affine utility support iter lowerbound and diagnostics
#9742 - [Schedule] Add Annotate/Unannotate primitive
#9738 - [TensorIR] Primitive "SetScope"
#9743 - [Schedule] Analysis functions to check if compute_inline and com…
#9689 - Allow memory (aka storage) scopes to be retrieved/applied to PrimFuncs
#9559 - [TensorIR][UX] Type annotation-based runtime type checking
#9444 - Add a 'rolling_buffer' scheduling primitive
#9360 - [TensorIR] Cross-Thread Reduction

TOPI

#11531 - TE implementation of LSTM using scan
#11161 - Add Adreno GPU target and topi supporting textures with dynamically allocated textures
#10332 - VNNI support for batch matmul
#9873 - Add support for groupped conv3d
#10230 - VNNI support for int8 dense
#10098 - [Op]5 ops can accept unsigned integers as indices
#9832 - Support grouped conv1d
#9694 - Add generic batch norm
#9233 - Cortex-M DSP support

TVMScript

#11308 - Represent ramp as index slice
#10099 - Support T.buffer_decl using data pointer from Let/Allocate
#9680 - Improve printer for TIR syntax sugar
#9492 - Add syntax sugar for T.handle and T.match_buffer
#9620 - Add for loop syntax sugar
#9543 - Misc error message improvements
#9505 - [Fix] Add type hints for more uncovered cases

USMP

#11015 - U3 use case
#10189 - Adding support for U1 usecase for constant pools
#10785 - Adding support for U4 usecase
#10193 - adding support for U2 and U3 usecases
#10005 - Add performance characteristics to PoolInfo
#9565 - [TIR]Integrating USMP to AoT Executor
#9704 - Hill Climb allocator
#9418 - [TIR]adding the pass to convert to pool offsets
#9649 - [TIR]Augmenting the algo interface with memory pressure
#9214 - [TIR]Greedy memory planning algorithm
#8468 - [TIR]Added buffer info extraction pass

microNPU

#11468 - Optimize separate padding operation for conv2d
#11453 - Add transform matrices and part matcher to identity op
#11410 - add E2E tests with cascader wo striping
#11288 - Expose compute cycle annotations to TIR lowering
#10959 - Add a pass to reorder copy and compute nodes
#10509 - Add various options to the cascader
#11263 - Adding a option to enable striping
#10251 - Add support for conv2d running on two cores on U65
#10862 - Integrate the cascader
#10344 - Integrate rolling buffers in Arm(R) Ethos(TM)-U
#10824 - Some housekeeping in the test_ethosu folder
#10763 - Tweak a layout transform matrix
#10725 - Add a pass to move allocate nodes to the outer scope
#10695 - Determine block configs using the cascader
#10599 - Refactor Relay to TIR hook
#10508 - Improve cascader memory transfer estimates
#10345 - Add support for TFLite FULLY_CONNECTED
#10254 - Introduce a pass to remove redundant identity operations
#10062 - [5] Convert Proposals to te.Schedules
#9959 - [4] Add the cascader Proposal generator
#10022 - enable USMP
#10127 - Add support for LeakyReLU
#10004 - Add FreeRTOS variant of NPU demo
#10060 - Refactor type inference data type checks
#9960 - Add support for pack and unpack
#10143 - Fix layout assignment in layout optimizer pass
#9890 - [3] Plan generation for the cascader
#9855 - Add support for transpose convolution
#9841 - Add support for nearest neighbor and bilinear upsampling
#9951 - Removing constant args from PrimFunc
#9929 - Refactor base address determination to codegen
#9910 - Add support for requantize
#9831 - Move optimization passes to be a module pass and ensure they are running
#9785 - [2d] Add more Part matchers to cascader
#9778 - [2c] Add performance modelling to cascader
#9471 - [2b] Create CascaderGraphs from TE graphs
#9469 - [2a] Add CascaderGraph for cascading analysis
#9621 - Add support for SPLIT and SPLIT_V
#9508 - Update Conv2D Tests to Use TF API to Gen Test Cases
#9627 - Add support for SIGMOID
#9589 - Add support for TFLite concatenate
#9623 - Refactor codegen tests
#9561 - Add NHWC -> NHCWB16 layout transformation pass
#9576 - Mean legalization support
#9597 - Move the compilation to use Target Hooks.
#9458 - [1] Add affine analysis structures for the cascader
#9547 - Add the infrastructure for lookup table and TANH
#9521 - Support binary elementwise with non-4D inputs
#9560 - Fix incorrectly calculated stride when converting NHWC to NHCWB16
#9530 - Add unary elementwise operator infrastructure with ABS
#9514 - Adding rounding mode attribute to operators
#9515 - Allow constants to be given as input to an operator

microTVM

#11250 - [ARM] Add Relay tests for conv2d registered schedules
#11232 - [rpc] Implemented rpc logging
#11044 - Add support for host-driven AoT Executor
#11043 - Better version handling for Arduino
#10555 - Enable micro tvmc tutorial testing in CI
#10194 - [RVM] Add scripts for automated build and testing
#10144 - TVMCon 2021 Zephyr Demo with CMSIS-NN
#10024 - [tvmc] Add TVMC Micro tutorial for Zephyr
#9684 - Fix zephye/test_zephyr_armv7m test
#9584 - [TVMC] Add TVMC test for Arduino and Zephyr
#9526 - Add minimal forwarding RPC server for host driven python execution on Hexagon
Zephyr support - #11362, #10138

Misc

#11465 - Add cooldown interval logic for the profiling functional
#11888 - [LLVM] Include LLVM headers in files that use them, not in llvm_common.h
#11646 - [Arith] Simplification of ceil, log2, and left_shift
#11464 - [MLF] Add support for multiple modules in Model Library Format
#11632 - [AutoTVM][Autoscheduler] Default build funcs inherit PassContext
#11543 - [OpenCL] Implement conv2d_winograd algorithm for Adreno
#11287 - [Arith] Merge surjective/non-surjective iter mapping detections
#11393 - Add utility to replace direct call to pytest.main
#11252 - [ROOFLINE] Roofline analysis over RPC
#11000 - [Graph Debugger] Expose way to benchmark individual nodes.
#10794 - bump PyTorch version to 1.11
#10821 - [REFACTOR] Remove legacy nnvm folder
#10798 - [Arith] Remove diagnostic ctx argument from DetectIterMap
#10567 - [Refactor] Reduced repetition in CodeGenLLVM's buffer access
#10455 - [AUTO_SCHEDULER] Add feature extraction directly from PrimFunc
#7401 - RFC: initial stab at TorchScript fallback
#10391 - [vulkan] Add integer dot product (4xint8, 4xuint8) tensorization for the vulkan SPIR-V target.
#10293 - [VirtualMachine] new method allowing to set one input tensor by its index or name
#10191 - Generate correct output tensor names in C Interface API
#9276 - Parameterize test_link_params
#9808 - [Rust] Update Rust bindings
#9553 - [PROFILING] Add ability to profile a single function_profiling
#9611 - [CMAKE] Automatically detect newly added source files
#9544 - [Target] enable -arch=sm_xx for assigning cuda target arch and deprecate autotvm.measure.set_cuda_target_arch api
Profiler - #11530, #11066
Docs - #10921, #11403, #10774, #10912, #9633, #9906, #9534, #9307, #9654, #9580
Android - #11241
ETHOSN - #11261, #10486, #10018, #9596
TVMC - #11012, #10962, #10722, #9817, #9529, #9229

Closing now that the vote thread has passed: #12133

repo: apache/tvm


text:
[Bug] RPCTimeEvaluator signature error

Expected behavior
The output should be something as follows that is shown in the user tutorial https://tvm.apache.org/docs/tutorial/cross_compilation_and_rpc.html#sphx-glr-tutorial-cross-compilation-and-rpc-py
1.389e-07 secs/op
Actual behavior
I got an error as shown below.
  File "/home/priyadrasta/rpc cross compilation.py", line 51, in <module>
    time_f = func.time_evaluator(func.entry_name, dev, number=10)
  File "/home/priyadrasta/.local/lib/python3.10/site-packages/tvm/runtime/module.py", line 335, in time_evaluator
    feval = _ffi_api.RPCTimeEvaluator(
  File "/home/priyadrasta/.local/lib/python3.10/site-packages/tvm/_ffi/_ctypes/packed_func.py", line 237, in __call__
    raise get_last_ffi_error()
tvm.error.RPCError: Traceback (most recent call last):
  11: TVMFuncCall
  10: _ZN3tvm7runtime13PackedFun
  9: tvm::runtime::TypedPackedFunc<tvm::runtime::PackedFunc (tvm::runtime::Optional<tvm::runtime::Module>, std::string, int, int, int, int, int, int, int, std::string)>::AssignTypedLambda<tvm::runtime::__mk_TVM0::{lambda(tvm::runtime::Optional<tvm::runtime::Module>, std::string, int, int, int, int, int, int, int, std::string)#1}>(tvm::runtime::__mk_TVM0::{lambda(tvm::runtime::Optional<tvm::runtime::Module>, std::string, int, int, int, int, int, int, int, std::string)#1}, std::string)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs const, tvm::runtime::TVMRetValue) const
  8: tvm::runtime::RPCModuleNode::GetTimeEvaluator(std::string const&, DLDevice, int, int, int, int, int, std::string const&)
  7: tvm::runtime::RPCWrappedFunc::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
  6: tvm::runtime::RPCClientSession::CallFunc(void*, TVMValue const*, int const*, int, std::function<void (tvm::runtime::TVMArgs)> const&)
  5: tvm::runtime::RPCEndpoint::CallFunc(void*, TVMValue const*, int const*, int, std::function<void (tvm::runtime::TVMArgs)>)
  4: tvm::runtime::RPCEndpoint::HandleUntilReturnEvent(bool, std::function<void (tvm::runtime::TVMArgs)>)
  3: tvm::runtime::RPCEndpoint::EventHandler::HandleNextEvent(bool, bool, std::function<void (tvm::runtime::TVMArgs)>)
  2: tvm::runtime::RPCEndpoint::EventHandler::HandleProcessPacket(std::function<void (tvm::runtime::TVMArgs)>)
  1: tvm::runtime::RPCEndpoint::EventHandler::HandleReturn(tvm::runtime::RPCCode, std::function<void (tvm::runtime::TVMArgs)>)
  0: _ZN3tvm7runtime6deta
  11: TVMFuncCall
  10: tvm::runtime::PackedFuncObj::Extractor<tvm::runtime::PackedFuncSubObj<tvm::runtime::__mk_TVM1::{lambda(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)#1}> >::Call(tvm::runtime::PackedFuncObj const*, tvm::runtime::__mk_TVM1, tvm::runtime::TVMRetValue)
  9: tvm::runtime::RPCServerLoop(int)
  8: tvm::runtime::RPCEndpoint::ServerLoop()
  7: tvm::runtime::RPCEndpoint::HandleUntilReturnEvent(bool, std::function<void (tvm::runtime::TVMArgs)>)
  6: tvm::runtime::RPCEndpoint::EventHandler::HandleNextEvent(bool, bool, std::function<void (tvm::runtime::TVMArgs)>)
  5: tvm::runtime::RPCEndpoint::EventHandler::HandleProcessPacket(std::function<void (tvm::runtime::TVMArgs)>)
  4: tvm::runtime::RPCSession::AsyncCallFunc(void*, TVMValue const*, int const*, int, std::function<void (tvm::runtime::RPCCode, tvm::runtime::TVMArgs)>)
  3: tvm::runtime::LocalSession::CallFunc(void*, TVMValue const*, int const*, int, std::function<void (tvm::runtime::TVMArgs)> const&)
  2: _ZN3tvm7runtime13PackedF
  1: tvm::runtime::TypedPackedFunc<tvm::runtime::PackedFunc (tvm::runtime::Optional<tvm::runtime::Module>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, int, int, int, int, int, int, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)>::AssignTypedLambda<tvm::runtime::__mk_TVM0::{lambda(tvm::runtime::Optional<tvm::runtime::Module>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, int, int, int, int, int, int, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)#1}>(tvm::runtime::__mk_TVM0::{lambda(tvm::runtime::Optional<tvm::runtime::Module>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, int, int, int, int, int, int, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)#1}, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMArgs const&) const
  0: _ZN3tvm7runtime6deta
  File "/workspace/tvm/src/runtime/rpc/rpc_endpoint.cc", line 376
RPCError: Error caught from RPC call:
[06:44:36] /home/pi/tvm/include/tvm/runtime/packed_func.h:1728: Function runtime.RPCTimeEvaluator(0: runtime.Module, 1: basic_string<char>, 2: int, 3: int, 4: int, 5: int, 6: int, 7: int, 8: int, 9: int, 10: basic_string<char>) -> runtime.PackedFunc expects 11 arguments, but 10 were provided.

Environment
I am using ubuntu 22.04 LTS as my local machine and Raspberry Pi 4 as my target remote device with an IP address : 192.168.91.173 and port: 9090
Steps to reproduce

build runtime on Raspberry Pi as shown in the tutorial
Set Up RPC Server on Raspberry Pi using the command : python -m tvm.exec.rpc_server --host 0.0.0.0 --port=9090
save the following code in local machine with LLVM enabled  as rpc cross compilation.py  and run :

import numpy as np

import tvm
from tvm import te
from tvm import rpc
from tvm.contrib import utils

n = tvm.runtime.convert(1024)
A = te.placeholder((n,), name="A")
B = te.compute((n,), lambda i: A[i] + 1.0, name="B")
s = te.create_schedule(B.op)

local_demo = False

if local_demo:
   target = "llvm"
else:
   target = "llvm -mtriple=aarch64-linux-gnu"

func = tvm.build(s, [A, B], target=target, name="add_one")
# save the lib at a local temp folder
temp = utils.tempdir()

path = temp.relpath("lib.tar")

func.export_library(path)

if local_demo:
   remote = rpc.LocalSession()
else:
   # The following is my environment, change this to the IP address of your target device
   host = "192.168.91.173"  #host IP address of my Raspberry Pi
   port = 9090
   remote = rpc.connect(host, port)

remote.upload(path)
func = remote.load_module("lib.tar")

# create arrays on the remote device
dev = remote.cpu()
a = tvm.nd.array(np.random.uniform(size=1024).astype(A.dtype), dev)
b = tvm.nd.array(np.zeros(1024, dtype=A.dtype), dev)
# the function will run on the remote device
func(a, b)
np.testing.assert_equal(b.numpy(), a.numpy() + 1)

time_f = func.time_evaluator(func.entry_name, dev, number=10)
cost = time_f(a, b).mean
print("%g secs/op" % cost)

I remember seeing this error, it was fixed by updating and recompiling the runtime code on the device. Have you updated the runtime on the device recently?

I am also getting this same error, I am using PYNQ Z2 with v2.7
tvm.error.RPCError: Traceback (most recent call last):
11: TVMFuncCall
10: _ZN3tvm7runtime13PackedFun
9: tvm::runtime::TypedPackedFunc<tvm::runtime::PackedFunc (tvm::runtime::Optionaltvm::runtime::Module, std::string, int, int, int, int, int, int, int, std::string)>::AssignTypedLambda<tvm::runtime::__mk_TVM0::{lambda(tvm::runtime::Optionaltvm::runtime::Module, std::string, int, int, int, int, int, int, int, std::string)#1}>(tvm::runtime::__mk_TVM0::{lambda(tvm::runtime::Optionaltvm::runtime::Module, std::string, int, int, int, int, int, int, int, std::string)#1}, std::string)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs const, tvm::runtime::TVMRetValue) const
8: tvm::runtime::RPCModuleNode::GetTimeEvaluator(std::string const&, DLDevice, int, int, int, int, int, std::string const&)
7: tvm::runtime::RPCWrappedFunc::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
6: tvm::runtime::RPCClientSession::CallFunc(void*, TVMValue const*, int const*, int, std::function<void (tvm::runtime::TVMArgs)> const&)
5: tvm::runtime::RPCEndpoint::CallFunc(void*, TVMValue const*, int const*, int, std::function<void (tvm::runtime::TVMArgs)>)
4: tvm::runtime::RPCEndpoint::HandleUntilReturnEvent(bool, std::function<void (tvm::runtime::TVMArgs)>)
3: tvm::runtime::RPCEndpoint::EventHandler::HandleNextEvent(bool, bool, std::function<void (tvm::runtime::TVMArgs)>)
2: tvm::runtime::RPCEndpoint::EventHandler::HandleProcessPacket(std::function<void (tvm::runtime::TVMArgs)>)
1: tvm::runtime::RPCEndpoint::EventHandler::HandleReturn(tvm::runtime::RPCCode, std::function<void (tvm::runtime::TVMArgs)>)
0: _ZN3tvm7runtime6deta
8: TVMFuncCall
7: tvm::runtime::PackedFuncObj::Extractor<tvm::runtime::PackedFuncSubObj<tvm::runtime::{lambda(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)#2}> >::Call(tvm::runtime::PackedFuncObj const*, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)
6: tvm::runtime::RPCServerLoop(int)
5: tvm::runtime::RPCEndpoint::ServerLoop()
4: tvm::runtime::RPCEndpoint::HandleUntilReturnEvent(bool, std::function<void (tvm::runtime::TVMArgs)>)
3: tvm::runtime::RPCSession::AsyncCallFunc(void*, TVMValue const*, int const*, int, std::function<void (tvm::runtime::RPCCode, tvm::runtime::TVMArgs)>)
2: tvm::runtime::LocalSession::CallFunc(void*, TVMValue const*, int const*, int, std::function<void (tvm::runtime::TVMArgs)> const&)
1: _ZN3tvm7runtime13PackedF
0: tvm::runtime::TypedPackedFunc<tvm::runtime::PackedFunc (tvm::runtime::Optionaltvm::runtime::Module, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, int, int, int, int, int, int, int, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator >)>::AssignTypedLambda<tvm::runtime::{lambda(tvm::runtime::Optionaltvm::runtime::Module, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, int, int, int, int, int, int, int, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator >)#1}>(tvm::runtime::{lambda(tvm::runtime::Optionaltvm::runtime::Module, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, int, int, int, int, int, int, int, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator >)#1}, std::__cxx11::basic_string<char, std::char_traits, std::allocator >)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs const, tvm::runtime::TVMRetValue) const
File "/workspace/tvm/src/runtime/rpc/rpc_endpoint.cc", line 376
RPCError: Error caught from RPC call:
[01:41:04] /home/xilinx/tvm/include/tvm/runtime/packed_func.h:1728: Function runtime.RPCTimeEvaluator(0: runtime.Module, 1: basic_string, 2: int, 3: int, 4: int, 5: int, 6: int, 7: int, 8: int, 9: int, 10: basic_string) -> runtime.PackedFunc expects 11 arguments, but 10 were provided.
I can see that the error is part of this function, but its not clear why?
/src/runtime/rpc/rpc_module.cc
TVM_REGISTER_GLOBAL("runtime.RPCTimeEvaluator")
.set_body_typed([](Optional opt_mod, std::string name, int device_type, int device_id,
int number, int repeat, int min_repeat_ms, int limit_zero_time_iterations,
int cooldown_interval_ms, int repeats_to_cooldown,
std::string f_preproc_name) {


text:
[LIBXSMM][BYOC] Integrate Libxsmm with BYOC

As mentioned in https://github.com/apache/tvm-rfcs/blob/main/rfcs/0046-Intel-LIBXSMM-integration.md#upstream-plan, this is the 2nd PR we propose to accelerate small gemm with libxsmm by BYOC.

@comaniac Could you help review this PR?

@tqchen @comaniac Could you help assign a reviewer for this PR? Thank you.

A gentle ping @comaniac

@comaniac Actually there is still a major issue hasn't been addressed in current my implementation: how to enable libxsmm only in the inference mode?
That's because libxsmm doesn't have a DENSE implementation, so we have to use GEMM instead which requires a explicit weight transpose. This won't be a problem in inference mode because the weight is frozen thus we can pre-transpose and store the transposed weight, no performance penaty at all.
But I failed to find out a way to check weather the functional-style relay graph is in the inference mode.  Do you have any suggestions?

I don't think Relay has a "training" mode...

I don't think Relay has a "training" mode...

Exactly. I don't even find a way to identify whether a tensor is constant at compile/run time. I think this is a serious problem for Relay which is a functional flavor language. Abstractions like "constant tensor" is missing here, which is actually ubiquitous in almost all DL frameworks.

Ah I see. You were saying whether the weight is a constant or a var.
Well, this is not about training or inference mode. Model definition is always separated with the model weight values, so when users convert a model to Relay, weights are vars. Since TVM is mainly for inference, it doesn't make sense to have "training" mode and I don't think it's a problem.
Meanwhile, Relay allows users to bind parameters with trained weights to let them become constants.
Since this is user options, as a codegen, you have deal with both cases. If weights are vars, then you have to transpose them in runtime; otherwise you could use constant updater to transpose them at compile time.

Ah I see. You were saying whether the weight is a constant or a var.
Well, this is not about training or inference mode. Model definition is always separated with the model weight values, so when users convert a model to Relay, weights are vars. Since TVM is mainly for inference, it doesn't make sense to have "training" mode and I don't think it's a problem.
Meanwhile, Relay allows users to bind parameters with trained weights to let them become constants. You should find the difference when traversing the graph. VarNode and ConstantNode are two different nodes and they are easy to be identified.
Since this is user options, as a codegen, you have deal with both cases. If weights are vars, then you have to transpose them in runtime; otherwise you could use constant updater to transpose them at compile time.

Cool, bind_params_by_name() does work! I will upstream my change soon, thank you!

@comaniac I just upstream the "constant_updater" related code, now the transposed weight is managed in codegen as expected.

authors: Hzfengsy comaniac zhuwenxi


text:
[CI] Update dummy-variable regex for pylint

Prior to this commit, the regex used for pylint to identify dummy variables would correctly identify variables that start with an underscore (e.g. _scale), unless they have an underscore elsewhere in the name (e.g. _scale_factor).  This leads to false positives from pylint for unused variables, as prefixing a variable with an underscore should mark a variable as intentionally unused.
This commit updates the regex in TVM's pylintrc to match the current default value for dummy-variables-rgx, to allow unused variables to be named with a leading underscore, even if they also contain another underscore.

I ran into this false positive for https://github.com/apache/tvm/pull/17201 for a variable named _in_group (screenshot below).  While I could work around it by renaming to _ingroup, it's also good to have a long-term fix for the lint rule.

So, my question is why do we want to name unused variables in general? Could we not just use _?

Mainly to indicate to a reader which return values are being ignored. As an example, suppose there's a function that produces a
# A reader may not know what the first two return values are.
_, _, attrs = define_function(relax_args)

# The first two return values are a function and parameters to
# that function.
_func, _params, attrs = define_function(relax_args)

I've run into a few cases where the same return value was re-generated, because a later developer wasn't aware that it was already being generated. By explicitly naming the unused return value, this becomes less likely. (e.g. Seeing _func instead of _, and checking whether it is already the function that should be used.)
Obviously, this shouldn't be applied in all cases, as _, y = get_xy() is already perfectly readable, but it can help in cases where it isn't as clear.

Makes sense, thanks for your reply.

authors: Lunderberg quic-sanirudh


text:
[ETHOSN] Add support for non-default ethos-n78 configurations

Updated tvmc with new ethos-n78 composite target.
Added additional n78 specific ethos-n configuration options.

cc @leandron @Leo-arm @u99127 @mbaret


text:
[MINOR] Rearrange the contents in "About Zeppelin"

What is this PR for?
Just rearranged the placement of Zeppelin logo, title and text in "About Zeppelin". In the mobile screen as well.
What type of PR is it?
Improvement
What is the Jira issue?
No Jira issue for this
How should this be tested?
To check this change in your local, build only zeppelin-web with ./grunt build and then browse with ./grunt serve.
Screenshots (if appropriate)

Before

desktop & tablet screen

mobile screen

After

desktop & tablet screen

mobile screen

Questions:

Does the licenses files need update? no
Is there breaking changes for older versions? no
Does this needs documentation? no

would Patua One font be not available on some platforms?

@felixcheung AFAIK since this Patua One is from here, if some browser/platform can't recognize this font, it means that browser/platform users can't see any "Apache Zeppelin" logo written with Patua One.
Then they can't see this one as well. Do we need to cover this case?

Hmmm I don't think anyone has reported this being an issue so I think we are good.
LGTM

One CI test is failed in SCALA_VER="2.11" SPARK_VER="2.0.0" HADOOP_VER="2.3" PROFILE="-Pspark-2.0 -Phadoop-2.3 -Ppyspark -Psparkr -Pscalding -Pexamples -Pscala-2.11" like below
Tests: succeeded 17, failed 1, canceled 0, ignored 0, pending 0
*** 1 TEST FAILED ***
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Zeppelin ........................................... SUCCESS [ 28.461 s]
[INFO] Zeppelin: Interpreter .............................. SUCCESS [ 30.071 s]
[INFO] Zeppelin: Zengine .................................. SUCCESS [ 11.502 s]
[INFO] Zeppelin: Display system apis ...................... FAILURE [ 20.268 s]
[INFO] Zeppelin: Spark dependencies ....................... SKIPPED
[INFO] Zeppelin: Spark .................................... SKIPPED
[INFO] Zeppelin: Markdown interpreter ..................... SKIPPED
[INFO] Zeppelin: Angular interpreter ...................... SKIPPED
[INFO] Zeppelin: Shell interpreter ........................ SKIPPED
[INFO] Zeppelin: Livy interpreter ......................... SKIPPED
[INFO] Zeppelin: HBase interpreter ........................ SKIPPED
[INFO] Zeppelin: PostgreSQL interpreter ................... SKIPPED
[INFO] Zeppelin: JDBC interpreter ......................... SKIPPED
[INFO] Zeppelin: File System Interpreters ................. SKIPPED
[INFO] Zeppelin: Flink .................................... SKIPPED
[INFO] Zeppelin: Apache Ignite interpreter ................ SKIPPED
[INFO] Zeppelin: Kylin interpreter ........................ SKIPPED
[INFO] Zeppelin: Python interpreter ....................... SKIPPED
[INFO] Zeppelin: Lens interpreter ......................... SKIPPED
[INFO] Zeppelin: Apache Cassandra interpreter ............. SKIPPED
[INFO] Zeppelin: Elasticsearch interpreter ................ SKIPPED
[INFO] Zeppelin: BigQuery interpreter ..................... SKIPPED
[INFO] Zeppelin: Alluxio interpreter ...................... SKIPPED
[INFO] Zeppelin: web Application .......................... SKIPPED
[INFO] Zeppelin: Server ................................... SKIPPED
[INFO] Zeppelin: Packaging distribution ................... SKIPPED
[INFO] Zeppelin: Scalding interpreter ..................... SKIPPED
[INFO] Zeppelin: Examples ................................. SKIPPED
[INFO] Zeppelin: Example application - Clock .............. SKIPPED
[INFO] Zeppelin: Example application - Horizontal Bar chart SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:32 min
[INFO] Finished at: 2016-09-29T07:00:40+00:00
[INFO] Final Memory: 48M/469M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.scalatest:scalatest-maven-plugin:1.0:test (test) on project zeppelin-display_2.11: There are test failures -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :zeppelin-display_2.11

and seems irrelevant with this change.

@AhyoungRyu Tested, LGTM!

@corneadoug Thanks! I retriggered CI just for sure. After checking CI then will merge this :)

@AhyoungRyu Rebasing from master should fix the CI issue

@corneadoug rebased but it still failed in Selenium test. The error log is like below
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 76.542 sec - in org.apache.zeppelin.integration.SparkParagraphIT

Results :

Failed tests:
  ParagraphActionsIT.testWidth:314 New Width is : 4
Expected: <true>
     but: was <false>
  ParagraphActionsIT.testWidth:314 New Width is : 8
Expected: <true>
     but: was <false>

Tests in error:
  ZeppelinIT.testSparkInterpreterDependencyLoading:234->AbstractZeppelinIT.waitForParagraph:70->AbstractZeppelinIT.pollingWait:96 Â» Timeout

Tests run: 16, Failures: 2, Errors: 1, Skipped: 0

[INFO]
[INFO] --- scalatest-maven-plugin:1.0:test (test) @ zeppelin-server ---
Discovery starting.
Oct 06, 2016 8:09:31 AM org.openqa.selenium.safari.SafariDriverServer start
INFO: Server started on port 13724
Oct 06, 2016 8:09:31 AM org.openqa.selenium.safari.SafariDriverCommandExecutor stop
INFO: Shutting down
Oct 06, 2016 8:09:31 AM org.openqa.selenium.safari.SafariDriverCommandExecutor stop
INFO: Stopping server
Oct 06, 2016 8:09:31 AM org.openqa.selenium.safari.SafariDriverServer stop
INFO: Stopping server
Oct 06, 2016 8:09:31 AM org.openqa.selenium.safari.SafariDriverCommandExecutor stop
INFO: Shutdown complete
Discovery completed in 4 seconds, 967 milliseconds.
Run starting. Expected test count is: 1
Zeppelin is already running
WelcomePageSuite:
- Welcome sign is correct
Zeppelin stop [  OK  ]
Run completed in 13 seconds, 782 milliseconds.
Total number of tests run: 1
Suites: completed 3, aborted 0
Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
All tests passed.
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (stop-zeppelin) @ zeppelin-server ---
[INFO] Executing tasks

main:
     [exec] Zeppelin is not running
[INFO] Executed tasks
[INFO]
[INFO] --- maven-failsafe-plugin:2.16:verify (default) @ zeppelin-server ---
[INFO] Failsafe report directory: /home/travis/build/apache/zeppelin/zeppelin-server/target/failsafe-reports
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Zeppelin: Interpreter .............................. SUCCESS [ 12.285 s]
[INFO] Zeppelin: Zengine .................................. SUCCESS [  6.291 s]
[INFO] Zeppelin: Display system apis ...................... SUCCESS [  3.479 s]
[INFO] Zeppelin: Spark dependencies ....................... SUCCESS [ 37.316 s]
[INFO] Zeppelin: Spark .................................... SUCCESS [  6.898 s]
[INFO] Zeppelin: Server ................................... FAILURE [09:22 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10:29 min
[INFO] Finished at: 2016-10-06T08:09:45+00:00
[INFO] Final Memory: 46M/376M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.16:verify (default) on project zeppelin-server: There are test failures.
[ERROR]
[ERROR] Please refer to /home/travis/build/apache/zeppelin/zeppelin-server/target/failsafe-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :zeppelin-server

Seems not irrelevant with this change :)

Merging this if there are no more discussions

Go for it

authors: AhyoungRyu corneadoug felixcheung

repo: apache/zeppelin

license_source: bigquery


text:
[MINOR] Job parameters synchronization

What is this PR for?
We need to synchronize setting up job result, exception, errorMessage and dateFinished in the end of Job.run() method. This will guarantee that we won't see these parameters in inconsistent state in multithreading environment, e.g., if we see that a dateFinished parameter has not been set up yet we will be sure that we also don't have a job final result.
This code refactoring will make it possible to resolve issue ZEPPELIN-1856.
What type of PR is it?
Refactoring
Todos

[ ] - Synchronize setResult(), exception, errorMessage and dateFinished in Job.run() method.
[ ] - Synchronize getters and setters for exception, errorMessage and dateFinished.

Questions:

Does the licenses files need update? no
Is there breaking changes for older versions? no
Does this needs documentation? no

Thanks for the contribution!
LGTM

Merge to master and branch-0.7 if no further comments.

Merged to master, but didn't merged to branch-0.7 because of conflict.

authors: AlexanderShoshin Leemoonsoo


text:
Out of date govmomi being used in libretto

Minimal changes required to make things work in libretto with the new govmomi
Changes Done:-
Context in import path updated. Go 1.7 moves the golang.org/x/net/context package into the standard library as context.
UnitNumber now returning reference, hence removing '&'
CreateDisk takes one more parameter, datastore ManagedObjectReference
Conflicting int and int32

so this needs a go vet fix, but it also can't build as the new govmomi depends on go 1.7 and the CI build uses an older version of go.
apcera people, have any comments?

I just submitted #116 to update the CI system to only test with Go 1.7 and Go 1.8. That PR should be merged shortly and then after you rebase that will hopefully unblock this PR. Thanks for your patience!

This was superceded by #120, which updates govmomi to the latest. Thank you for your patience!

repo: apcera/libretto


text:
Evaluate Togglz integration with CAS

Resource

https://www.togglz.org

Ideas
Each feature module in CAS would turn into a Toggle, that can then be used in the /info endpoint to describe the current feature-set of CAS?

Evaluated. Simply not feasible, and could not find a decent relevant use case. Closing for now. Can revisit later.

repo: apereo/cas


text:
Use authn attribute release policy for resource password grant type

Backport of: https://github.com/apereo/cas/pull/5527

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

authors: CLAassistant leleuj


text:
[PRO] EOL: update warming runtime to nodejs14.x

Prerequisites

[x] I am running the latest version. (up upgrade)
[x] I searched to see if the issue already exists.
[x] Are you an Up Pro subscriber?

Description
AWS started sending out mails about nodejs10.x reaching EOL. The warming runtime cannot be changed by up users. Please upgrade to nodejs14.x (no need to go for 12 I guess).

Refs #784
I just realized that a project I recently upgraded to Node.js 14 still uses Node.js 8 for warming. What do I need to do to update the warming lambda? I was expecting up deploy production to handle that. Please don't tell me I need to remove warming from the config, deploy and then add it again?

Hello! To update the warming functions you may need to delete and run up stack plan/apply again, those are part of the "stack". It should be fine to leave them as-is though, even though it's EOL, AWS will keep existing functions running

Thanks!

It should be fine to leave them as-is though, even though it's EOL, AWS will keep existing functions running

Same comment as https://github.com/apex/up/issues/784#issuecomment-544510618 :smile: . I'm not the one getting the e-mails, but my clients. But I'll communicate that this does not affect the services.

Has this been fixed? I am running the latest 1.7.1-pro version, but when I ran up, I still got this error:
Error: deploying: ap-southeast-2: The following resource(s) failed to create: [WarmingFunction].

Hi @tj, just wonder if this warming function will be upgraded to nodejs14.x soon. I think that's a very useful feature of the pro version. However it is not working anymore for new applications since AWS has deprecated old node version. An update would be really appreciated.

Bump to this request- this is one of the most important features that causes me to use the Pro version, please let us know when you'll be able to upgrade the warming function runtime?

The "[Action Required] AWS Lambda end of support for Node.js 12" e-mails are rolling out, so we already missed one cycle.

authors: Prinzhorn codebravotech mytechtip tj

repo: apex/up

license: MIT


text:
Bar chart custom tooltip function is called repeatedly on mouseover

In the below demo, move the mouse over a column and look at the top left corner at the number of events fired:
https://codepen.io/vsync/pen/jgmxGb
I would expect it to be called only on mouseenter event and not thousands on mousemove which affects performance.
Is there any justification for the current behaviour?

The tooltip is a common functionality between line/area/scatter/bar/column charts and the mousemove event is helpful in determining the values based on mouse position. As in the line/area charts, there are no elements where mouseenter/mouseleave can be captured,
mousemove is used for all chart types.

Bummer

authors: junedchhipa yairEO


text:
wrong xaxis labels generation

Codepen
https://codepen.io/y_nk/pen/MWgMGNP
Explanation
When using series with [x, y] point format, xaxis labels are calculated in a wrong way.
In this codepen, you can see :
dates in a generated array for 2019-10-01 to 2019-10-07
serie is a pseudo random generated serie with data points for 2019-10-01, 2019-10-03 and 2019-10-05
In the current codepen, there's no use for dates for now, but ideally it should be provided in xaxis.categories. regardless of this, xaxis labels are wrongly generated. You can see the logs i've provided in the <pre> block and compare the x values in serie and what is given as val to xaxis.labels.formatter

What is the behavior you expect?
ideally, xaxis.categories is taken in consideration for xaxis.type === 'datetime' so we can use dates instead of auto-generated labels.

What is happening instead?
labels are wrongly generated.

What error message are you getting?
none

If you want complete control over datetime generation, along with the formatter function, you can use xaxis.min, xaxis.tickAmount and xaxis.max like below
xaxis: {
  type: 'datetime',
  min: new Date('2019-10-01 GMT').getTime(),
  max: new Date('2019-10-10 GMT').getTime(),
  tickAmount: 10,
  labels: {
    formatter: (val, series, index) => {
      return new Date(val).toString()
    }
  }
}

authors: junedchhipa y-nk


text:
Release V3 alpha/beta

I'm at a point where I can't stay on starknet v5 anymore, I have to upgrade to v6.
But starknet-react is stuck on v5, hooks do not work if my project resolves starknet v6.
Please publish the @starknet-react/core next-v3 branch as alpha or beta!

Hello, yes we finished migrating all (most) hooks to the new version and are going to release the next one. Curious to hear why you cannot use starknet v5 anymore? AFAIK it should not break until all the ecosystem migrated to starknet v6.

I'm working with Dojo, and dojo.js already migrated to v6, preparing to add Paymaster support on execute calls.
With v5, I start getting errors, because they already use some new types.
With v6, some of your core hooks stop working.
I'm stuck, about to recreate some of your hooks just so I can continue working.
That's why I'm asking to release an alpha.

Do you need to support wallets like braavos and argent or you have a smart wallet? I'm happy to push a pre-release this week but in that case you'll need to implent your own connector to work with browser wallets (looks like braavos already works with v6 tho).

That sounds good. I'll start making pre-releases of v3 but wallet support is still not 100% complete. Worst case someone will have to implement an adapter for the v6 to v5 wallet api.

jumping into this discussion, i have also faced issues with Argent when upgrading v5 -> v6. So far braavos is working well.

I started testing wallets this week, useBlockNumber()` also has some issues, and I get all sort of weird errors.
A pre-release would be good!

Hello, I started publishing pre-releases (3.0.0-beta.0 is the current one). I'm testing it with the most recent versions of braavos and argent.

Hi @fracek is there any docs we can start looking?

The preview is available here. Will add instructions on how to update shortly.

Hi Team, any ETA of the stable version? Starknet-react is the core of our project.
You are an amazing team, thank you so much for all your effort!

@gianalarcon  Sorry for the late reply, I missed this comment! You can already upgrade by switching to the @next tag. Everything is implemented and only small bugs need to be ironed out. Waiting for the full ecosystem to update before making a final release.

gm Team, wondering when will the @next version have a stable released  ? @fracek 👀

There's still a couple of small items related to in-app wallets before we tag the release as latest. Want to avoid breaking changes after it's an official release.

Hi @fracek , I was wondering whether the version on the main branch is the intended stable version? if not do you have an estimate time of the release? I've noticed significant performance drops on the v3 account connections compared to its predecessors.

@fracek thanks for the demo! very helpful!
As for how i concluded the perf issue, I noticed that when trying to connect, it takes a while for update the connected state of the wallet (from useAccount, disconnected to connected). I've also noticed that argentX is worse in the duration of connection. This did not happen in the previous version. Do you have any indication of potential issues and how to fix?
Slow argentX:

The issue seems to be related to the new wallet.request API. I will double check if there's also any issue on our side, but if the issue is slow response from wallets there's not much we can do.

@fracek Thanks for the hard work!

authors: edulanasca fracek gianalarcon jrcarlos2000 metalboyrick rsodre

repo: apibara/starknet-react


text:
Swagger UI gives no zlib library and no iconv library warnings

To repro:

git clone git@github.com:apigee-127/swagger-tools.git
cd swagger-tools/examples/2.0
npm install
node index.js
Surf to http://localhost:3000/docs/

You should get something along
 base is http://localhost:3000/docs/
shred.bundle.js:1726 no zlib library
shred.bundle.js:1734 no iconv library
swagger-ui.js:71 Loaded SwaggerUI

Tested in Chrome and Firefox.
The UI runs but it might be nice to have the warnings fixed as they might hide some issues.

This should be reported in the swagger-ui project as those lines are logged from it.

authors: bebraw whitlockjc

license: mit


text:
Request access to closed API

EPIC for closing APIs so that API is published in catalog, but it only shows limited information and has a e.g. button for the developer / user to request access to the API or a method for API owner to invite users to the API.
This task requires feedback from the team on possible options to allow only certain users to see/use the API with the smallest possible implementation.
Field level visibility of the API in catalog.
Group of allowed users
Method for getting the access: email invititation from owner, administraor can manually add users (by email address) or request button.

Feedback from @philippeluickx Does this require a new user role: API user that is tied to  particular API(s) to which user has been granted access?

We might want to create a seperate epic with user roles functionality. This requires the definition and management of roles and we have to implement permissions as well.

1.0  Only allow API owner to  limit catalogue visibility for users on Apinf side.
Estimation: 5
(Possible before MIndTrek)
2.0 Allove API owner to control developers who can make calls on proxy level:
Estimation: ?
Study story needed.

authors: bajiat philippeluickx preriasusi

repo: apinf/api-umbrella-dashboard


text:
stopped  components being built

stopped code from building  &  components that did nothing.

Hi, this PR seems to be removing all stateless components in the app, but this contradicts what we're doing. We'd like to give users the ability to nest stateless components in other stateful or stateless components. Can you please explain in greater detail what you are trying to fix?

what this does is that it stops it from writing  as this does nothing it doesnt stop normal components from rendering just he random components that do nothing

do this
app
home
hero
img
title
about
hero
see what happens when you run it normally compared to what mine does

In your example, can you be more specific with which components are stateful/stateless and what the nesting should look like (parent-child relationships)?
Thank you

authors: The-Code-Monkey menothe

repo: apjs/ReactVelocity


text:
using tagOrID for canvas

Hi,
I'm trying your package in my distribution and it's very easy to work with and works well with my version of Tk (8.7).
Would it be possible to use a canvas tag as widgetpath?
many thanks in advance,
best regards

Implemented in v1.2.0
Seen in demo
https://github.com/aplsimple/baltip/releases/download/baltip-1.2/baltip-1.2.mp4

it works like a charm.
thanks
++

repo: aplsimple/baltip


text:
Just a check

Hey @apolikar11 . how are you?

I am good thank you. test

test 2. ok?

repo: apolikar/Test


text:
How can I access the environmental variables in the resolvers

I'm trying to access the environmental variables here
`const typeDefs = gql'
type NewToken {
token: String
uuid: String
}
type Query {
getDbToken: NewToken
}
';
type NewToken = {
token: String
uuid: String
}
export default {
resolvers: {
Query: {
async getToken(env: any) {
console.log(env); // I'm getting undefined
var token = 'token2';
var uuid = 'uuid2';
try {
return { token: token, uuid: uuid } as NewToken;
} catch (error) {
console.log(error);
throw error;
}
}
}
},
typeDefs: [typeDefs]
};
`

I've found this from Apollo server docs
"A resolver can optionally accept four positional arguments: (parent, args, contextValue, info)"


text:
Provider Interface Support SPI & Optimize ProviderManager SPI

What's the purpose of this PR

Add SPI feature  like ApplicationProvider NetworkProvider in DefaultProviderManager
Add Ordered in ProviderManager and use ServiceBootstrap.loadPrimary instead of com.ctrip.framework.foundation.internals.ServiceBootstrap#loadFirst.   Because getManager method in class Foundation  use com.ctrip.framework.foundation.internals.ServiceBootstrap#loadFirst to load interface ProviderManager implementation, if I want to define my ownProviderManager, which class has higher priority can not spefic.

Brief changelo
Provider Interface Support SPI & Optimize ProviderManager SPI
Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Read the Contributing Guide before making this pull request.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Write necessary unit tests to verify the code.
[x] Run mvn clean test to make sure this pull request doesn't break anything.
[ ] Update the CHANGES log.

I have read the CLA Document and I hereby sign the CLA

updated

Codecov Report

Merging #4218 (8473f51) into master (ced8ad9) will increase coverage by 0.00%.
The diff coverage is 80.00%.

@@            Coverage Diff            @@
##             master    #4218   +/-   ##
=========================================
  Coverage     52.58%   52.58%
- Complexity     2631     2632    +1
=========================================
  Files           485      486    +1
  Lines         15225    15226    +1
  Branches       1573     1573
=========================================
+ Hits           8006     8007    +1
  Misses         6663     6663
  Partials        556      556

Impacted Files
Coverage Δ

...trip/framework/foundation/spi/ProviderManager.java
0.00% <0.00%> (ø)

...ava/com/ctrip/framework/foundation/Foundation.java
38.70% <100.00%> (ø)

...internals/provider/DefaultApplicationProvider.java
49.16% <100.00%> (ø)

...nfigservice/filter/ClientAuthenticationFilter.java
93.61% <0.00%> (+2.12%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ced8ad9...8473f51. Read the comment docs.

authors: codecov-commenter darcydai

repo: apolloconfig/apollo


text:
Note breaking change of network interface

TODO:

[ ] If this PR is a new feature, reference an issue where a consensus about the design was reached (not necessary for small changes)
[ ] Make sure all of the significant new logic is covered by tests
[ ] Rebase your changes on master so that they can be merged easily
[ ] Make sure all tests and linter rules pass
[ ] Update CHANGELOG.md with your change
[ ] Add your name and email to the AUTHORS file (optional)
[ ] If this was a change that affects the external API, update the docs and post a link to the PR in the discussion

@helfer do we actually want to make this change? It doesn't seem to really improve much, just seems like a breaking change for the sake of it. Perhaps we can keep backcompat and print a warning, then remove it in 1.0?

@stubailo I made the change because there was a comment in the code that said we were just doing that for backcompat. I think now is as good a time as any to make that breaking change, but I'm not dead set on it, if you want to undo it.

I guess we never documented the new way and it's really easy to not break it - so maybe let's put it back?

authors: helfer stubailo

repo: apollostack/apollo-client


text:
New: Support merging tags

Description
Add support for merging policy tags into runtime to ensure trireme can echo back attributes.

Codecov Report

Merging #415 into master will decrease coverage by 0.03%.
The diff coverage is 44.44%.

@@            Coverage Diff             @@
##           master     #415      +/-   ##
==========================================
- Coverage   56.05%   56.02%   -0.04%
==========================================
  Files          45       45
  Lines        6099     6103       +4
==========================================
  Hits         3419     3419
- Misses       2373     2377       +4
  Partials      307      307

Impacted Files
Coverage Δ

policy/runtime.go
67.56% <0%> (-2.53%)
:arrow_down:

configurator/configurator.go
39.79% <80%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0a5da9b...170be3c. Read the comment docs.

authors: codecov-io satyamsi

repo: aporeto-inc/trireme-lib


text:
Fix issue introduced in apostrophe 2.102.0 where newly inserted pieces are unpublished

... Unless the published property is explicitly passed, etc. A major bug in practice.

Requires the corresponding Apostrophe PR to solve the problem although it won't crash without it.

repo: apostrophecms/apostrophe-headless


text:
update to use rspec 3

These changes (mostly from @atambo) make the gem run cleanly for spec 3.

This will probably break older Rails. Does Apotomo still work with Cells 4? I haven't used it in ages!

No, I don't think apotomo works with Cells 4 (at least the gemspec doesn't allow it to). We basically just got our test suite passing with these changes by trying to follow the same changes that were made to rspec-cells.

@apotonick, would you be cool with merging these changes and bumping the version to 0.10 and doing a release? That way people can still keep to 0.9 for rspec 2 and 0.10 would be rspec 3?

authors: apotonick atambo kognate

repo: apotonick/rspec-apotomo


text:
Odd error message

When comparing a []byte to nil via BeCalled(With(...)) we got this odd error message:
Test Panicked
          runtime error: comparing uncomparable type []uint8
          /usr/local/go/src/runtime/panic.go:423

          Full Stack Trace
          /usr/local/go/src/runtime/alg.go:250 (0x2f2a)
                efaceeq: panic(errorString("comparing uncomparable type " + *t._string))
          /Users/bob/gobal/src/github.com/apoydence/eachers/becalled.go:65 (0x134f73)
                (*Called).Match: if call[i] != recvd {
          /Users/bob/gobal/src/github.com/onsi/gomega/internal/asyncassertion/async_assertion.go:110 (0x1ea98a)
          /Users/bob/gobal/src/github.com/onsi/gomega/internal/asyncassertion/async_assertion.go:48 (0x1e9e98)
          /Users/bob/gobal/src/foo/bar.go:77 (0x87870)
                glob.func1.2.2.2.3: Eventually(mockPublisher.PublishInput.Value).Should(BeCalled(With(bytes)))

Not very obvious from the error message what the problem is.

I fact this test fails even if a valid []byte is read from mockPublisher.PublishInput.Value.

repo: apoydence/eachers


text:
Clarifications on dependency matching and split between fetching and getting.

The spec in the "Dependency matching" chapter says that every dependency should be discovered using the ACI discovery process and then, just the downloaded image should be verified with the requested labels or matching ImageID.
But, in appc/spec#16 this process is split in two phases: ACI fetching and ACI rendering.
I think that the splitting between the fetching phase and the rendering phase is good. Someone can, for example, not fetch the image from a repository but just import an aci manually in the store. Or there can be an offline/caching mode.
So the ACI rendering will be executed independently from the ACI fetching.
In the ACI rendering phase the ACI will be requested to the store providing an imageID or an app name and some labels. In the latter case the store can find more than one ACI that matches the requested App name and labels.
Which one should be returned?
I have some possible cases in mind:
Case 1
If I'm requesting to the store an App name without any label, all the ACIs with that App Name will match your request.
Which one should be returned?
Should it be implementation dependent or should it be defined in the spec?
The current implementation i tried to do in coreos/rocket#297 uses the latest aci imported in the store.
Case 2
Another example can be the "latest" pattern: If I'm requesting to the store an App without any version label, how can I satisfy the "latest" pattern?

Fetch phase: In this phase, as I'm not passing a specific version label, the aci discovery will download something like https://example.com/appname-latest-linux-amd64.aci. On the http server it can be a link to an aci called appname-2.0.1-linux-amd64.aci and in its manifest the version label will be "2.0.1".

Rendering phase: I'm requesting from the store an ACI named appname and without any version label. The store can have other ACIs previously downloaded or imported for this app name with different version (for example "1.0.0" "strangereleasename" etc...).
As the version label is just a string without any SemVer or lexical ordering rule there isn't an ordering in these labels.

As in [Case 1](#Case 1) the implementation i tried to do in coreos/rocket#297 chooses the latest imported ACI but prefers ACIs imported with the "latest" flag (its saved in an appinfo additional index):
On fetching, if downloaded with the "latest" pattern, the ACI is saved in the store with the "latest" flag. When requesting an aci without specifying a version label the ones with the latest flag are preferred over the others (or perhaps only the one with the latest flag should be choosed?).

Are my thoughts correct?
If so, there should be possible ACI choosing logics other then the one I tried to implement? Should this logics be clarified in the SPEC?
If so, should the spec be clarified (also with some examples)?

As all the patches implementing the above behavior are landed, I assume that it's the wanted behavior (at least for rocket).
Now the question is:

is this implementation dependent (as rocket uses the cas as a local cache) and the SPEC shouldn't be changed but perhaps some examples should be added to the Implementation Guide.

or

the SPEC should clarify the statement in https://github.com/appc/spec/blob/master/SPEC.md#dependency-matching introducing the splitting between fetching and getting.

I think I would argue that the local cache (CAS, in Rocket's case) is essentially just another an implementation of the discovery service, at least in terms of the dependency matching - so "fetching" and "getting"  are essentially the same operation in this sense, and the behaviour is implementation dependent.
We should definitely address this more clearly in the spec, though. @sgotti if this makes sense to you do you have something in mind you could put up as a PR?

authors: jonboulle sgotti

repo: appc/spec