Let's be honest, Jive (this forum the Community uses) is not a code sharing platform or documentation repository. Yet, sprinkled throughout this community there are awesome discussions, snippets of code, queries, tutorials, demos, videos, and screenshots, that even the search feature fail to bubble up when needed. And still, scattered out there on the rest of the internet (with repos on Github and GIFs on Twitter) are more contributions that are tucked away without a spotlight.
I've been trying to tackle this for awhile, and had put it on the back burner, because there just wasn't a good solution. Meanwhile, I was working on this which resulted in ccsd/ledbelly, which I released after coming back from paternity leave. With our new Live Events Daemon, I decided to tackle improving our Canvas Data pipeline of which is also a community contribution and has run very well for 3 years, thanks to James Jones. In evaluating the current community options for managing Canvas Data and trying to decide which one I was going to implement and/or contribute to... I finally took my head out of the Canvas box and searched the internet for "importing big data into sql" or something to that effect and I found onurakpolat/awesome-bigdata: A curated list of awesome big data frameworks, resources and other awesomeness. This is where I found Embulk, and why I've shared Managing Canvas Data with Embulk. About a week after finding Embulk, I found Fluentd, on the same list, which his a similar plug-in/out solution by the same group which has the potential for handling real-time Live Events... or SQS to SQL, in case that's something you're interested in.
Embulk has 43 contributors, dozens of plugins, recent updates, and has been around longer than Canvas Data.
I haven't tested Fluentd, because I like our LED, and writing it is teaching me Ruby, but really wish I'd found Fluentd first.
Maybe when the Live Events has a better beta environment configuration I'll give it a test.
You may have noticed a pattern here, a duplication of efforts, a lack of understanding the broader context of the problems at hand, knowing what to Google, with a very focused goal of solving the problem locally. As a web developer, I've handled CSV import and exports for years, these were never terabyte+ collections. I didn't have the context, understanding, or experience to know that I should be solving the problem of Big Data, I just wanted CSV files in SQL so I could write web applications. We find ourselves here, in a community, often after changing roles and job titles, sometimes we know what we're doing, sometimes we don't.
Reinventing wheels is expensive and time consuming. Let's start doing less work, and documenting what does.
The CanvasLMS Awesome List
Our Awesome List, is a collaborative resource owned by the entire Canvas Community. The list is here to highlight the generous contributions which stand out, solve unique problems, and provide detailed information for the benefit of all. You can contribute by sharing another member's awesome contribution, or your own.
And don't forget to visit the root of all Awesome Lists at awesome.re, there's a topic for you.
awesome/awesome.md at master · sindresorhus/awesome · GitHub
The awesome manifesto
...try to only include actual awesome stuff in your list. After all, it's a curation, not a collection.
But what is awesome?
Only awesome is awesome
Research if the stuff you're including is actually awesome. Only put stuff on the list that you or another contributor can personally recommend. You should rather leave stuff out than include too much.
We'll be editing a public document and elevating some content above others.
Before contributing it's important to review the Canvas Community guidelines
Jive doesn't have pull requests or commit history like Github, where most awesome lists are located, and putting our Awesome List on Github would limit our community members ability to contribute without having a Github account. Documents in Jive are versioned, and this offers the ability to edit, and track changes in case of errors and accountability.
Adding something to the awesome list
- Before editing, make sure someone isn't already editing
No idea if this would actually be a problem, but let's give people some unpressured time to finish their contribution
- Click the Edit link at the top right of the document
- Adding a topic
If a topic does not exist you can add it.
Keep the formatting of the document as a list, with topic titles using an h2 and solid grey bottom border.
An empty paragraph between topics helps with spacing.
The HTML will look like this. If you'd like help adding a new topic, just ask.
<h2 style="border-bottom: 1px solid grey;"><a name="NewTopic"></a>New Topic</h2>
<li>New Topic Item</li>
For new topics, we'd also like an anchor link in the Contents section at the top.
<li><a href="#NewTopic">New Topic</a></li>
- Adding an Item
- Using the RCE, find where your item belongs, place your cursor at the end of the previous item and hit [enter]
- Copy the link in, hopefully Jive will pull the page title, if not rename it to the title of the resource
- Add a description, generally whatever description is provided, or summarize it
- Curating the list
As the Awesome Manifesto states this is a curation, not a collection.
From time to time it may be necessary to update the list by removing outdated and older contributions with the new, as well as replacing items which may be better or more awesome than previous contributions. This is OK, this is what the list of resources is for. Sorting Dashboard Course Cards was AWESOME, but we don't need it anymore. We are always grateful for the contributions, but when it's no longer necessary or relevant or there is something better, it's simply an update, not a critique.
Sometimes discussion and review may be necessary before adding or removing items, please use the comments section of CanvasLMS Awesome List Contributions for this purpose.
- Organizing the list
There's a soft alphabetical organization, but you'll also notice it kind of goes from basic to advanced topics.