Opensource.com https://opensource.com/ en How upstream contributions power scientific research https://opensource.com/article/23/2/open-science-upstream-contributions <span class="field field--name-title field--type-string field--label-hidden">How upstream contributions power scientific research</span> <span class="field field--name-uid field--type-entity-reference field--label-hidden"><a title="View user profile." href="/users/cdelia" class="username">cdelia</a></span> <span class="field field--name-created field--type-created field--label-hidden">Fri, 02/03/2023 - 03:00</span> <div class="clearfix text-formatted field field--name-body field--type-text-with-summary field--label-hidden field__item"><p>Horizon Europe emphasizes open science and open source technology. The program evolved from Horizon 2020, which provided financial support for research projects that promoted industrial competitiveness, advanced scientific excellence, or solved social challenges through the process of "open science."</p> <p>Open science is an approach to the scientific process based on open cooperative work, tools, and diffusing knowledge found in the <a href="https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/agr-contr/unit-mga_he_en.pdf" target="_blank">Horizon Europe Regulation and Model Grant Agreement</a>. This open science approach aligns with open source principles that provide a structure for such cooperation.</p> <p>The open source principles are:</p> <ul><li>Transparency</li> <li>Collaboration</li> <li>Release early, release often</li> <li>Inclusion</li> <li>Community orientation</li> </ul><p>In creating open source software, one of the basic foundational principles of open source software development is an "upstream first" philosophy. The opposite direction is "downstream," and upstream and downstream make up the ecosystem for a given software package or distribution. Upstreams are important because that's where the source contribution comes from.</p> <p></p><div class="embedded-resource-list callout-float-right"> <div class="field field--name-title field--type-string field--label-hidden field__item">Open science and sustainability</div> <div class="field field--name-links field--type-link field--label-hidden field__items"> <div class="field__item"><a href="https://www.redhat.com/en/creating-chris?intcmp=7016000000127cYAAQ">Video series: ChRIS (ChRIS Research Integration System)</a></div> <div class="field__item"><a href="https://research.redhat.com/?intcmp=7016000000127cYAAQ">Explore Red Hat Research projects</a></div> <div class="field__item"><a href="https://opensource.com/article/22/12/open-source-sustainability?intcmp=7016000000127cYAAQ">6 articles to inspire open source sustainability</a></div> <div class="field__item"><a href="https://opensource.com/article/22/4/how-linux-saves-earth?intcmp=7016000000127cYAAQ">How Linux rescues slow computers (and the planet)</a></div> <div class="field__item"><a href="https://opensource.com/tags/science?intcmp=7016000000127cYAAQ">Latest articles about open science</a></div> <div class="field__item"><a href="https://opensource.com/tags/education?intcmp=7016000000127cYAAQ">Latest articles about open education</a></div> <div class="field__item"><a href="https://opensource.com/tags/sustainability?intcmp=7016000000127cYAAQ">Latest articles about sustainability</a></div> </div> </div> <p>Each upstream is unique, but generally, the upstream is where decisions are made and where the community for a project collaborates for the project's objectives. Work done upstream can flow out to many other open source projects. The upstream is also a place where developers can report bugs and security vulnerabilities. If a bug or security flaw is fixed upstream, then every downstream project or product based on the upstream can benefit from that work.</p> <p>It is important to contribute to the work side-by-side with the rest of the community from which you benefit. By working upstream first, there is the opportunity to vet ideas with the larger community and work together to build new features, releases, content, etc. It's far better if all the contributors work together rather than contributors from different companies, universities, or affiliations working on features behind closed doors and then trying to integrate them later. Open source contributions can outlive the research project duration making a more durable impact.</p> <p>As an example of such contributions, in the <a href="https://cordis.europa.eu/project/id/609828" target="_blank">ORBIT FP7 EU</a> project, a feature was developed by Red Hat (lower layers, such as Linux Kernel and QEMU) and Umea University (upper layers, such as LibVirt and OpenStack) and contributed to their related upstream communities. This enabled "<a href="https://wiki.qemu.org/Features/PostCopyLiveMigration" target="_blank">post-copy live migration of VMs</a>" in OpenStack. Even though that was done several years ago, that feature is still available (and independently maintained) in any OpenStack distribution today (as well as plain LibVirt and QEMU).</p> <p>Just as with software development, research under Horizon Europe promotes the adoption of sharing research outputs as early and widely as possible to citizen science, developing new indicators for evaluation research, and rewarding researchers. With open source upstream communities, the research contributed can extend beyond the research project timeline by feeding into the upstream life cycle. This allows future consumption by companies, universities, governments, etc., to evolve and further secure the research's project contribution.</p> <hr /><p><span style="font-size:11pt; font-variant:normal; white-space:pre-wrap"><span style="font-family:'Open Sans',sans-serif"><span style="color:#222222"><span style="font-weight:400"><span style="font-style:normal"><span style="text-decoration:none">This article originally appeared on <a href="https://www.icos-project.eu/impact-of-upstreaming-research-contributions" target="_blank">The Impact of Upstreaming Research Contributions</a> and is republished with permission.</span></span></span></span></span></span></p> </div> <div class="clearfix text-formatted field field--name-field-article-subhead field--type-text-long field--label-hidden field__item"><p>Just as with software development, research under Horizon Europe promotes the adoption of sharing research outputs as early and widely as possible to citizen science, developing new indicators for evaluation research, and rewarding researchers.</p> </div> <div class="field field--name-field-lead-image field--type-entity-reference field--label-hidden field__item"><article class="media media--type-image media--view-mode-caption"> <div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="/sites/default/files/lead-images/OSCDC_Chemistry_OpenScience_520x292_12268077_0614MM.png" width="520" height="292" alt="Open science rocket and beaker" title="Open science rocket and beaker" /> </div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>Opensource.com</p> </div> </article> </div> <div class="field field--name-field-tags field--type-entity-reference field--label-hidden field__items"> <div class="field__item"><a href="/tags/science" hreflang="en">Science</a></div> <div class="field__item"><a href="/tags/openstack" hreflang="en">OpenStack</a></div> <div class="field__item"><a href="/tags/community-management" hreflang="en">Community management</a></div> </div> <div class="hidden field field--name-field-listicle-title field--type-string field--label-hidden field__item">What to read next</div> <div class="field field--name-field-default-license field--type-list-string field--label-hidden field__item"><a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/"> <img alt="Creative Commons License" src="/themes/osdc/assets/img/cc-by-sa-4.png" title="This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License." /></a>This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.</div> <article class="profile"> <a href="/points-and-badges" class="profile__points">870 points</a> <div class="profile__container"> <div class="field field--name-field-user-picture field--type-entity-reference field--label-hidden field__item"><article class="media media--type-image media--view-mode-no-captions"> <div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="/sites/default/files/2023-01/Hawthorn.JPG" width="3840" height="5760" alt="Leslie Hawthorn headshot" /> </div> </article> </div> <div class="field field--name-field-location field--type-string field--label-hidden field__item">Germany</div> <div class="clearfix text-formatted field field--name-field-user-about field--type-text-with-summary field--label-hidden field__item"><p>Leslie Hawthorn has spent her career creating and cultivating open source communities. She has driven open source strategy in Fortune 10 companies, pre-IPO startups, and Foundation Boards including senior roles at Red Hat, Google, the Open Source Initiative, and Elastic. She currently leads the industry verticals community strategy team within Red Hat’s Open Source Program Office. She advocates for creating citizen-centric Smart Cities underpinned by open source and open standards, and spends her spare time on tech for social good projects. Born and raised in Silicon Valley, she has called Europe home for the past seven years and resides in Bonn.</p> </div> <div class="field field--name-field-twitter-id field--type-string field--label-hidden profile__twitter">| Follow lhawthorn</div> <div class="field field--name-field-linkedin-id field--type-string field--label-hidden profile__linkedin">| Connect lesliehawthorn</div> </div> <div class="profile__badges"> <a href="/points-and-badges" class="field field--name-name field--type-string field--label-above profile__badges__name profile__badges__color-">Open Source Evangelist</a> <a href="/points-and-badges" class="field field--name-name field--type-string field--label-above profile__badges__name profile__badges__color-">Author</a> <a href="/resources/contributor-club" class="field field--name-name field--type-string field--label-above profile__badges__name profile__badges__color-">Contributor Club</a> </div> </article> <article class="profile"> <a href="/points-and-badges" class="profile__points">31 points</a> <div class="profile__container"> <div class="field field--name-field-user-picture field--type-entity-reference field--label-hidden field__item"><article class="media media--type-image media--view-mode-no-captions"> <div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="/sites/default/files/2023-01/jose_profile.jpg" width="3761" height="3762" alt="Profile pic" /> </div> </article> </div> <div class="field field--name-field-location field--type-string field--label-hidden field__item">Spain</div> <div class="clearfix text-formatted field field--name-field-user-about field--type-text-with-summary field--label-hidden field__item"><p>Have been designing and implementing IaaS/PaaS solutions, namelly OpenStack and Kubernetes/OpenShift, for the last 8 years, and teaching postgraduate courses for the last 7 years.</p> </div> <div class="field field--name-field-linkedin-id field--type-string field--label-hidden profile__linkedin">| Connect jose-castillo-lema</div> </div> <div class="profile__badges"> <a href="/points-and-badges" class="field field--name-name field--type-string field--label-above profile__badges__name profile__badges__color-">Open Enthusiast</a> <a href="/points-and-badges" class="field field--name-name field--type-string field--label-above profile__badges__name profile__badges__color-">Author</a> </div> </article> <section id='comments' class="field field--name-field-comments field--type-comment field--label-hidden comment-wrapper"> <div class="comments__count"> <div class="login"><a href="/user/register?absolute=1">Register</a> or <a href="/user/login?destination=/feed&amp;absolute=1">Login</a> to post a comment.</div> </div> </section> Fri, 03 Feb 2023 08:00:00 +0000 cdelia 70435 at https://opensource.com https://opensource.com/article/23/2/open-science-upstream-contributions#comments How I apply open source principles to filmmaking https://opensource.com/article/23/2/open-filmmaking <span class="field field--name-title field--type-string field--label-hidden">How I apply open source principles to filmmaking</span> <span class="field field--name-uid field--type-entity-reference field--label-hidden"><a title="View user profile." href="/users/psubhashish" class="username">psubhashish</a></span> <span class="field field--name-created field--type-created field--label-hidden">Fri, 02/03/2023 - 03:00</span> <div class="clearfix text-formatted field field--name-body field--type-text-with-summary field--label-hidden field__item"><p>As a nonfiction filmmaker, I have made over nine films, all under open licenses. But that choice always comes at a cost. It's tricky, if not impossible, to release a film under an open license if the film's copyright owner does not fully own the footage used. Films often purchase rights for media produced by others to be able to use in their work legally. As my films are mostly in endangered or low-resource languages with little or no pre-existing media, the option for purchasing existing media is often out of the question.</p> <p>On the other hand, film productions often record hours of footage and audio but only use a small percentage of those in the film. Footage that might not have an immediate use for a film's production house can be useful for others. In my case, many interviewed marginalized communities have a moral ownership over the footage. But researchers and others who interview them do not always provide communities with direct and open access. For these ethical and practical reasons, it is a good idea to share footage and the film under an open license and inform the communities interviewed or featured.</p> <p>There are many reasons filmmakers cannot release films under an open license, but this post is for those who somehow can. I often imagined what the open source equivalent would be for films that adhere to the Openness philosophy. Enter "Open Filmmaking," a framework that encourages releasing the source code of a film, i.e., footage under open licenses, and actively uses other practices such as open source software and <a href="https://opensource.com/article/16/12/yearbook-top-open-source-creative-tools-2016" target="_blank">open multimedia resources</a>.</p> <p>In my two recent documentary film projects, "The Volunteer Archivists" and "Nani Ma," I utilized media with open licenses and various <a href="https://opensource.com/article/22/2/open-source-creative-apps" target="_blank">open source software</a> (FLOSS). The films explore the areas of citizen science, archiving public domain text, documentation of oral history, and the use of open licenses and open source, as well as volunteerism.</p> <h2>The Volunteer Archivists</h2> <p>"<a href="https://theofdn.org/film/tva" target="_blank">The Volunteer Archivists</a>" follows the sixteen-year journey of digital archiving of texts by a volunteer-led group called Srujanika from Bhubaneswar in the Indian state of Odisha. Founded in 1983 by a scientist couple, they have managed with a small workforce to archive over 10,000 volumes of books, magazines, and other periodicals published in the Odia language since the early 1800s. They now host the archived texts online at <a href="https://odiabibhaba.in/" target="_blank">OdiaBibhaba.in</a>. Their original work includes growing a citizen science community over two decades to promote popular science education outside textbooks, publishing a monthly magazine called "Bigyana Tarang," and several illustrated publications. As scanners became more affordable, they began the process of archiving in 2006, starting with the "Purnnachandra Odia Bhashakosha," a seven-volume lexicon from 1930-1940 that powers the Odia Wiktionary. Srujanika also contributed to the Wikimedia movement by localizing computing terms into Odia and creating a <a href="https://archive.org/details/computer-translation-style-convention-guide-for-odia" target="_blank">manual for Odia computer translation style and convention</a>. Linux distributions and FLOSS, like LibreOffice, were localized in Odia because of their effort.</p> <h2>Nani Ma</h2> <p>"<a href="https://theofdn.org/film/nanima/" target="_blank">Nani Ma</a>" is based on oral history that was never recorded in audiovisual media, narrated by the late Musamoni Panigrahi in an early 1900s register of the Baleswari/northern dialect of Odia. Musamoni was my grandmother, and it was quite late by the time I realized how unique her stories, songs, and narration style were. The register, storytelling, and overall oral history are important pieces of history as they were strongly influenced by the <a href="https://en.wikipedia.org/wiki/Orissa_famine_of_1866" target="_blank">Orissa famine of 1866</a>, a direct impact of the British colonization of India.</p> <p>The entire footage and supporting multimedia files used in "Nani Ma" are now available on the <a href="https://archive.org/details/nani-ma" target="_blank">Internet Archive</a> under a CC BY-SA 4.0 License, and the film will be available to the general public after screening in film festivals.</p> <p>In addition to using openly-licensed media and releasing the production media (video footage, audio recordings, still images, and promotional graphics) from these two films under open licenses, they also saw the use of several FLOSS. Some of the software includes <a href="https://opensource.com/article/20/4/audacity" target="_blank">Audacity</a> for audio editing and codec conversion, HandBrake for video conversions, extensive use of <a href="https://opensource.com/article/21/12/linux-draw-inkscape" target="_blank">Inkscape</a> and <a href="https://opensource.com/life/16/6/tricks-gimp-beginners" target="_blank">GIMP</a>, respectively, for all vector and raster image editing, <a href="https://opensource.com/article/21/12/desktop-publishing-scribus" target="_blank">Scribus</a> for typesetting documents, and typefaces under <a href="https://opensource.org/licenses/OFL-1.1" target="_blank">Open Font License</a> (available on Google Fonts and other places). I also used openly-licensed images from <a href="https://commons.wikimedia.org/wiki/Main_Page" target="_blank">Wikimedia Commons</a> and audio from <a href="https://freesound.org/" target="_blank">freesound.org</a>.</p> <p>Open Filmmaking can be fun and challenging at the same time.</p> <p></p><div class="embedded-resource-list callout-float-right"> <div class="field field--name-title field--type-string field--label-hidden field__item">Open multimedia and art resources</div> <div class="field field--name-links field--type-link field--label-hidden field__items"> <div class="field__item"><a href="https://www.redhat.com/sysadmin/music-video-linux-terminal?intcmp=7016000000127cYAAQ">Music and video at the Linux terminal</a></div> <div class="field__item"><a href="https://opensource.com/article/22/2/open-source-creative-apps?intcmp=7016000000127cYAAQ">26 open source creative apps to try this year</a></div> <div class="field__item"><a href="https://www.redhat.com/en/open-source-stories?intcmp=7016000000127cYAAQ">Film series: Open Source Stories</a></div> <div class="field__item"><a href="https://opensource.com/downloads/blender-hotkey-cheat-sheet?intcmp=7016000000127cYAAQ">Blender cheat sheet</a></div> <div class="field__item"><a href="https://opensource.com/downloads/kdenlive-cheat-sheet?intcmp=7016000000127cYAAQ">Kdenlive cheat sheet</a></div> <div class="field__item"><a href="https://opensource.com/content/cheat-sheet-gimp?intcmp=7016000000127cYAAQ">GIMP cheat sheet</a></div> <div class="field__item"><a href="https://opensource.com/tags/audio-and-music?intcmp=7016000000127cYAAQ">Latest audio and music articles</a></div> <div class="field__item"><a href="https://opensource.com/tags/video-editing?intcmp=7016000000127cYAAQ">Latest video editing articles</a></div> </div> </div> <h2>Wrap up</h2> <p>Recommendations based on technical, ethical, and licensing constraints:</p> <ul><li>Most open source software often comes with beta releases which are many steps ahead of their stable releases. While they can add a modern flare to work, they aren't tested like the stable versions are. Use the stable version for mission-critical projects.</li> <li>Save your work-in-progress frequently.</li> <li>If you're new to open source creative tools, then budget time to learn new software.</li> <li>Budget time to create metadata. Audiovisual media without metadata is hard to find.</li> <li>Upload media publicly if you have consent from the people featured. B-rolls could sometimes include private data in the audio (as well as in the video). Redact such information before uploading. Uploading raw and unedited video featuring people always needs scrutiny.</li> <li>Use a Creative Commons license for releasing audiovisual content you own. But the <a href="https://creativecommons.org/licenses/" target="_blank">license spectrum</a> can be confusing. Use a tool like <a href="https://chooser-beta.creativecommons.org/" target="_blank">License Chooser</a> to evaluate what license makes the most sense.</li> <li>Use open codecs and other open multimedia resources while preparing files for uploading.</li> </ul><p>Lastly, finding a public hosting platform that shares the Openness movement's values can be intimidating. Internet Archive is my personal choice as it is helpful to create Collections under which files of different kinds can be uploaded (see "<a href="https://archive.org/details/the-volunteer-archivists" target="_blank">The Volunteer Archivists</a>" and "<a href="https://archive.org/details/nani-ma" target="_blank">Nani Ma</a>" collections). Happy open-filmmaking!</p> </div> <div class="clearfix text-formatted field field--name-field-article-subhead field--type-text-long field--label-hidden field__item"><p>Open Filmmaking is a framework that encourages releasing the source code of a film, i.e., footage under open licenses, and actively uses other practices such as open source software and open multimedia resources.</p> </div> <div class="field field--name-field-lead-image field--type-entity-reference field--label-hidden field__item"><article class="media media--type-image media--view-mode-caption"> <div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="/sites/default/files/lead-images/LIFE_film.png" width="520" height="292" alt="An old-fashioned video camera" title="An old-fashioned video camera" /> </div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>Opensource.com</p> </div> </article> </div> <div class="field field--name-field-tags field--type-entity-reference field--label-hidden field__items"> <div class="field__item"><a href="/tags/video-editing" hreflang="en">Video editing</a></div> <div class="field__item"><a href="/tags/art-and-design" hreflang="en">Art and design</a></div> <div class="field__item"><a href="/tags/open-studio" hreflang="en">Open Studio</a></div> <div class="field__item"><a href="/tags/licensing" hreflang="en">Licensing</a></div> </div> <div class="hidden field field--name-field-listicle-title field--type-string field--label-hidden field__item">What to read next</div> <div class="field field--name-field-default-license field--type-list-string field--label-hidden field__item"><a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/"> <img alt="Creative Commons License" src="/themes/osdc/assets/img/cc-by-sa-4.png" title="This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License." /></a>This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.</div> <section id='comments' class="field field--name-field-comments field--type-comment field--label-hidden comment-wrapper"> <div class="comments__count"> <div class="login"><a href="/user/register?absolute=1">Register</a> or <a href="/user/login?destination=/feed&amp;absolute=1">Login</a> to post a comment.</div> </div> </section> Fri, 03 Feb 2023 08:00:00 +0000 psubhashish 70432 at https://opensource.com https://opensource.com/article/23/2/open-filmmaking#comments Learn Basic by coding a game https://opensource.com/article/23/2/learn-basic-coding-game <span class="field field--name-title field--type-string field--label-hidden">Learn Basic by coding a game</span> <span class="field field--name-uid field--type-entity-reference field--label-hidden"><a title="View user profile." href="/users/moshez" class="username">Moshe Zadka</a></span> <span class="field field--name-created field--type-created field--label-hidden">Thu, 02/02/2023 - 03:00</span> <div class="clearfix text-formatted field field--name-body field--type-text-with-summary field--label-hidden field__item"><p>Writing the same application in multiple languages is a great way to learn new ways to program. Most programming languages have certain things in common, such as:</p> <ul><li>Variables</li> <li>Expressions</li> <li>Statements</li> </ul><p>These concepts are the basis of most programming languages. Once you understand them, you can start figuring the rest out.</p> <p>Programming languages usually share some similarities. Once you know one programming language, you can learn the basics of another by recognizing its differences.</p> <p>Practicing with a standard program is a good way of learning a new language. It allows you to focus on the language, not the program's logic. I'm doing that in this article series using a "guess the number" program, in which the computer picks a number between one and 100 and asks you to guess it. The program loops until you guess the number correctly.</p> <p>This program exercises several concepts in programming languages:</p> <ul><li>Variables</li> <li>Input</li> <li>Output</li> <li>Conditional evaluation</li> <li>Loops</li> </ul><p>It's a great practical experiment to learn a new programming language. This article focuses on Basic.</p> <h2>Guess the number in (Bywater) Basic</h2> <p>There is no real standard for the Basic programming language. Wikipedia says, "BASIC (Beginners' All-purpose Symbolic Instruction Code) is a family of general-purpose, high-level programming languages designed for ease of use." The <a href="https://yeolpishack.net/repos/ChipMaster/bwBASIC" target="_blank">BWBasic</a> implementation is available under the GPL.</p> <p>You can explore Basic by writing a version of the "guess the number" game.</p> <h2>Install Basic on Linux</h2> <p>In Debian or Ubuntu, you can install Basic with the following:</p> <pre> <code class="language-shell">$ apt install -y bwbasic</code></pre><p>Download the latest release tarball for Fedora, CentOS, Mageia, and any other Linux distribution. Extract it, make it executable, and then run it from a terminal:</p> <pre> <code class="language-shell">$ tar --extract --file bwbasic*z $ chmod +x bywater $ ./bywater </code></pre><p>On Windows, <a href="https://github.com/nerun/bwbasic/releases" target="_blank">download the .exe release</a>.</p> <h2>Basic code</h2> <p>Here is my implementation:</p> <pre> <code class="language-text">10 value$ = cint(rnd * 100) + 1 20 input "enter guess"; guess$ 30 guess$ = val(guess$) 40 if guess$ &lt; value$ then print "Too low" 50 if guess$ &gt; value$ then print "Too high" 60 if guess$ = value$ then 80 70 goto 20 80 print "That's right"</code></pre><p></p><div class="embedded-resource-list callout-float-right"> <div class="field field--name-title field--type-string field--label-hidden field__item">Programming and development</div> <div class="field field--name-links field--type-link field--label-hidden field__items"> <div class="field__item"><a href="https://developers.redhat.com/?intcmp=7016000000127cYAAQ">Red Hat Developers Blog</a></div> <div class="field__item"><a href="https://opensource.com/downloads/cheat-sheets?intcmp=7016000000127cYAAQ">Programming cheat sheets</a></div> <div class="field__item"><a href="https://www.redhat.com/en/services/training/learning-subscription?intcmp=7016000000127cYAAQ">Try for free: Red Hat Learning Subscription</a></div> <div class="field__item"><a href="https://opensource.com/downloads/bash-programming-guide?intcmp=7016000000127cYAAQ">eBook: An introduction to programming with Bash</a></div> <div class="field__item"><a href="https://developers.redhat.com/cheat-sheets/bash-shell-cheat-sheet?intcmp=7016000000127cYAAQ">Bash shell scripting cheat sheet</a></div> <div class="field__item"><a href="https://developers.redhat.com/e-books/modernizing-enterprise-java?intcmp=7016000000127cYAAQ">eBook: Modernizing Enterprise Java</a></div> <div class="field__item"><a href="https://opensource.com/downloads/building-applications?intcmp=7016000000127cYAAQ">An open source developer's guide to building applications</a></div> </div> </div> <p>Basic programs can be numbered or unnumbered. Usually, it is better to write programs unnumbered, but writing them with numbered lines makes it easier to refer to individual lines.</p> <p>By convention, coders write lines as multiples of 10. This approach allows interpolating new lines between existing ones for debugging. Here's an explanation of my method above:</p> <ul><li>Line 10: Computes a random value between 1 and 100 using the built-in <strong>rnd</strong> function, which generates a number between 0 and 1, not including 1.</li> <li>Line 20: Asks for a guess and puts the value in the <strong>guess$ scalar</strong> variable. Line 30 converts the value to a numeric one.</li> <li>Lines 40 and 50: Give the guesser feedback, depending on the comparison.</li> <li>Line 70: Goes to the beginning of the loop.</li> <li>Line 60: <em>Breaks</em>&amp; the loop by transferring control to line 80. Line 80 is the last line, so the program exits after that.</li> </ul><h2>Sample output</h2> <p>The following is an example of the program after putting it in <code>program.bas</code>:</p> <pre> <code class="language-shell">$ bwbasic program.bas  Bywater BASIC Interpreter/Shell, version 2.20 patch level 2 Copyright (c) 1993, Ted A. Campbell Copyright (c) 1995-1997, Jon B. Volkoff enter guess? 50 Too low enter guess? 75 Too low enter guess? 88 Too high enter guess? 80 Too low enter guess? 84 Too low enter guess? 86 Too high enter guess? 85 That's right</code></pre><h2>Get started</h2> <p>This "guess the number" game is a great introductory program for learning a new programming language because it exercises several common programming concepts in a pretty straightforward way. By implementing this simple game in different programming languages, you can demonstrate some core concepts of the languages and compare their details.</p> <p>Do you have a favorite programming language? How would you write the "guess the number" game in it? Follow this article series to see examples of other programming languages that might interest you!</p> </div> <div class="clearfix text-formatted field field--name-field-article-subhead field--type-text-long field--label-hidden field__item"><p>This tutorial lets you explore Basic by writing a version of the &quot;guess the number&quot; game.</p> </div> <div class="field field--name-field-lead-image field--type-entity-reference field--label-hidden field__item"><article class="media media--type-image media--view-mode-caption"> <div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="/sites/default/files/lead-images/laptop_screen_desk_work_chat_text.png" width="1040" height="584" alt="Person using a laptop" title="Person using a laptop" /> </div> </article> </div> <div class="field field--name-field-tags field--type-entity-reference field--label-hidden field__items"> <div class="field__item"><a href="/tags/programming" hreflang="en">Programming</a></div> </div> <div class="hidden field field--name-field-listicle-title field--type-string field--label-hidden field__item">What to read next</div> <div class="field field--name-field-default-license field--type-list-string field--label-hidden field__item"><a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/"> <img alt="Creative Commons License" src="/themes/osdc/assets/img/cc-by-sa-4.png" title="This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License." /></a>This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.</div> <section id='comments' class="field field--name-field-comments field--type-comment field--label-hidden comment-wrapper"> <div class="comments__count"> <h2>1 Comment</h2> <div class="login"><a href="/user/register?absolute=1">Register</a> or <a href="/user/login?destination=/feed&amp;absolute=1">Login</a> to post a comment.</div> </div> <article data-comment-user-id="126046" id="comment-220084" class="comment js-comment"> <mark class="hidden" data-comment-timestamp="1675346149"></mark> <footer class="comment__meta"> <article class="media media--type-image media--view-mode-byline"> <div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="/sites/default/files/styles/medium/public/osdc_default_avatar_1.png?itok=G0WcUo3c" width="100" height="100" alt="Avatar" title="Avatar" class="image-style-medium" /> </div> </article> <div> <div class="comment__submitted"> <a href="/users/jim-hall">Jim Hall</a> | February 2, 2023 <div class="rate-widget thumbsup"><form class="comment-content-220084-updown-rate-widget-0 vote-form" id="rate-widget-base-form" data-drupal-selector="comment-content-220084-updown-rate-widget-0" action="/feed" method="post" accept-charset="UTF-8"> <div class="rate-widget-thumbs-up"> <div class="thumbsup-rating-wrapper rate-disabled" can-edit="false"><div class="form-item js-form-item form-type-radio js-form-type-radio form-item-value js-form-item-value form-disabled"> <label class="form-item js-form-item form-type-radio js-form-type-radio form-item-value js-form-item-value form-disabled rating-label thumbsup-rating-label thumbsup-rating-label-up vote-pending rate-thumbs-up-btn-up"><input twig-suggestion="rating-input" class="rating-input thumbsup-rating-input form-radio" data-drupal-selector="edit-value-1" disabled="disabled" type="radio" id="edit-value-1" name="value" value="1" /> </label> </div> </div><input autocomplete="off" data-drupal-selector="form-umoybykf28d3-4sssdwbttniuu31ig5pnjrkkrfaz4u" type="hidden" name="form_build_id" value="form-umOybYkF28d3-4SSsDWBttniuU31iG5pNJRKKrfAZ4U" /> <input data-drupal-selector="edit-comment-content-220084-updown-rate-widget-0" type="hidden" name="form_id" value="comment_content_220084_updown_rate_widget_0" /> <input class="thumbsup-rating-submit button button--primary js-form-submit form-submit" data-drupal-selector="edit-submit" type="submit" id="edit-submit--2" name="op" value="Save" /> <div class="vote-result js-form-wrapper form-wrapper" data-drupal-selector="edit-result" id="edit-result"> <div title="Register or Login to like."><a href="/user/register">Register</a> or <a href="/user/login?destination=/feed">Login</a> to like</div> </div> <div class="rate-score"></div> </div> </form> </div> </div> <div class="content"> <div class="clearfix text-formatted field field--name-comment-body field--type-text-long field--label-hidden field__item"><p>Excellent article! I'll add that for anyone who wants to try this on FreeDOS, the Bywater BASIC package is included in the FreeDOS 1.3 Bonus installation CD.</p> </div> <drupal-render-placeholder callback="flag.link_builder:build" arguments="0=comment&amp;1=220084&amp;2=report_as_spam" token="8GA446apPd5XngeJaLHWpBjG71tAgcOUoPjXJxzdjAc"></drupal-render-placeholder><drupal-render-placeholder callback="comment.lazy_builders:renderLinks" arguments="0=220084&amp;1=full&amp;2=en&amp;3=" token="7pvMRr6RHr6EZflvZUtF30azJHFLwswzcOf9MWy8Q34"></drupal-render-placeholder> </div> </div> </footer> </article> </section> Thu, 02 Feb 2023 08:00:00 +0000 Moshe Zadka 70420 at https://opensource.com https://opensource.com/article/23/2/learn-basic-coding-game#comments A guide to fuzzy queries with Apache ShardingSphere https://opensource.com/article/23/2/fuzzy-query-apache-shardingsphere <span class="field field--name-title field--type-string field--label-hidden">A guide to fuzzy queries with Apache ShardingSphere</span> <span class="field field--name-uid field--type-entity-reference field--label-hidden"><a title="View user profile." href="/users/xionggaoxiang" class="username">xionggaoxiang</a></span> <span class="field field--name-created field--type-created field--label-hidden">Wed, 02/01/2023 - 03:00</span> <div class="clearfix text-formatted field field--name-body field--type-text-with-summary field--label-hidden field__item"><p><a href="https://shardingsphere.apache.org/" target="_blank">Apache ShardingSphere</a> is an open source distributed database and an ecosystem users and developers need for their databases to provide a customized and cloud-native experience. Its <a href="https://opensource.com/article/23/1/apache-shardingsphere-new-features" target="_blank">latest release</a> contains many new features, including data encryption integrated with existing SQL workflows. Most importantly, it allows fuzzy queries of the encrypted data.</p> <h2>The problem</h2> <p>By parsing a user's SQL input and rewriting the SQL according to the user's encryption rules, the original data is encrypted and stored with ciphertext data in the underlying database simultaneously.</p> <p>When a user queries the data, it fetches the ciphertext data from the database, decrypts it, and returns the decrypted original data to the user. However, because the encryption algorithm encrypts the whole string, users cannot run fuzzy queries.</p> <p>Nevertheless, many businesses need fuzzy queries after the data is encrypted. In version 5.3.0, Apache ShardingSphere provides users with a default fuzzy query algorithm that supports encrypted fields. The algorithm also supports hot plugging, which users can customize. The fuzzy query can be achieved through configuration.</p> <h2>How to achieve fuzzy query in encrypted scenarios</h2> <h3>Load data to the in-memory database (IMDB)</h3> <p>First, load all the data into the IMDB to decrypt it. Then, it'll be like querying the original data. This method can achieve fuzzy queries. If the amount of data is small, this method will prove simple and cost-effective. However, if the quantity of data is large, it'll be a disaster.</p> <h3>Implement encryption and decryption functions consistent with database programs</h3> <p>The second method is to modify fuzzy query conditions and use the database decryption function to decrypt data first and then implement fuzzy query. This method's advantage is the low cost of implementation, development, and use.</p> <p>Users only need to modify the previous fuzzy query conditions slightly. However, the ciphertext and encryption functions are stored together in the database, which cannot cope with the problem of account data leaks.</p> <p>Native SQL:</p> <pre> <code class="language-sql">select * from user where name like "%xxx%"</code></pre><p>After implementing the decryption function:</p> <pre> <code class="language-sql">ѕеlесt * frоm uѕеr whеrе dесоdе(namе) lіkе "%ххх%"</code></pre><h3>Store after data masking</h3> <p>Implement data masking on ciphertext and then store it in a fuzzy query column. This method could lack precision.</p> <p>For example, mobile number <strong>13012345678</strong> becomes <strong>130****5678</strong> after the masking algorithm is performed.</p> <h3>Perform encrypted storage after tokenization and combination</h3> <p>This method performs tokenization and combination on ciphertext data and then encrypts the resultset by grouping characters with fixed length and splitting a field into multiple ones. For example, we take four English characters and two Chinese characters as a query condition: <strong>ningyu1</strong> uses the four-character as a group to encrypt, so the first group is <strong>ning</strong>, the second group <strong>ingy</strong>, the third group <strong>ngyu</strong>, the fourth group <strong>gyu1</strong>, and so on. All the characters are encrypted and stored in the fuzzy query column. If you want to retrieve all data that contains four characters, such as <strong>ingy</strong>, encrypt the characters and use a key <code>like"%partial%"</code> to query.</p> <p>Shortcomings:</p> <ol><li>Increased storage costs: Free grouping will increase the amount of data, and the data length will increase after being encrypted.</li> <li>Limited length in fuzzy query: Due to security issues, the length of free grouping cannot be too short or the <a href="https://www.techtarget.com/whatis/definition/rainbow-table" target="_blank">rainbow table</a> will easily crack it. Like the example I mentioned above, the length of fuzzy query characters must be greater than or equal to four letters/digits or two Chinese characters.</li> </ol><p></p><div class="embedded-resource-list callout-float-right"> <div class="field field--name-title field--type-string field--label-hidden field__item">Our favorite resources about open source</div> <div class="field field--name-links field--type-link field--label-hidden field__items"> <div class="field__item"><a href="https://opensource.com/downloads/cheat-sheet-git?intcmp=7016000000127cYAAQ">Git cheat sheet</a></div> <div class="field__item"><a href="https://developers.redhat.com/cheat-sheets/advanced-linux-commands/?intcmp=7016000000127cYAAQ">Advanced Linux commands cheat sheet</a></div> <div class="field__item"><a href="https://opensource.com/tags/alternatives?intcmp=7016000000127cYAAQ">Open source alternatives</a></div> <div class="field__item"><a href="https://www.redhat.com/en/services/training/rh024-red-hat-linux-technical-overview?intcmp=7016000000127cYAAQ">Free online course: RHEL technical overview</a></div> <div class="field__item"><a href="https://opensource.com/downloads/cheat-sheets?intcmp=7016000000127cYAAQ">Check out more cheat sheets</a></div> </div> </div> <h3>Single-character digest algorithm (default fuzzy query algorithm provided in ShardingSphere version 5.3.0)</h3> <p>Although the above methods are all viable, it's only natural to wonder if there's a better alternative. In our community, we find that single-character encryption and storage can balance performance and query but fails to meet security requirements.</p> <p>So what's the ideal solution? Inspired by masking algorithms and cryptographic hash functions, we find that data loss and one-way functions can be used.</p> <p>The cryptographic hash function should have the following four features:</p> <ol><li>It should be easy to calculate the hash value for any given message.</li> <li>It should be difficult to infer the original message from a known hash value.</li> <li>It should not be feasible to modify the message without changing the hash value.</li> <li>There should only be a very low chance that two different messages produce the same hash value.</li> </ol><p>Security: Because of the one-way function, it's impossible to infer the original message. To improve the accuracy of the fuzzy query, we want to encrypt a single character, but the rainbow table will crack it.</p> <p>So we take a one-way function (to ensure every character is the same after encryption) and increase the frequency of collisions (to ensure every string is <strong>1: N</strong> backward), which greatly enhances security.</p> <h2>Fuzzy query algorithm</h2> <p>Apache ShardingSphere implements a universal fuzzy query algorithm using the below single-character digest algorithm <code>org.apache.shardingsphere.encrypt.algorithm.like.CharDigestLikeEncryptAlgorithm</code>.</p> <pre> <code class="language-plaintext">public final class CharDigestLikeEncryptAlgorithm implements LikeEncryptAlgorithm&lt;Object, String&gt; { private static final String DELTA = "delta"; private static final String MASK = "mask"; private static final String START = "start"; private static final String DICT = "dict"; private static final int DEFAULT_DELTA = 1; private static final int DEFAULT_MASK = 0b1111_0111_1101; private static final int DEFAULT_START = 0x4e00; private static final int MAX_NUMERIC_LETTER_CHAR = 255; @Getter private Properties props; private int delta; private int mask; private int start; private Map&lt;Character, Integer&gt; charIndexes; @Override public void init(final Properties props) { this.props = props; delta = createDelta(props); mask = createMask(props); start = createStart(props); charIndexes = createCharIndexes(props); } private int createDelta(final Properties props) { if (props.containsKey(DELTA)) { String delta = props.getProperty(DELTA); try { return Integer.parseInt(delta); } catch (NumberFormatException ex) { throw new EncryptAlgorithmInitializationException("CHAR_DIGEST_LIKE", "delta can only be a decimal number"); } } return DEFAULT_DELTA; } private int createMask(final Properties props) { if (props.containsKey(MASK)) { String mask = props.getProperty(MASK); try { return Integer.parseInt(mask); } catch (NumberFormatException ex) { throw new EncryptAlgorithmInitializationException("CHAR_DIGEST_LIKE", "mask can only be a decimal number"); } } return DEFAULT_MASK; } private int createStart(final Properties props) { if (props.containsKey(START)) { String start = props.getProperty(START); try { return Integer.parseInt(start); } catch (NumberFormatException ex) { throw new EncryptAlgorithmInitializationException("CHAR_DIGEST_LIKE", "start can only be a decimal number"); } } return DEFAULT_START; } private Map&lt;Character, Integer&gt; createCharIndexes(final Properties props) { String dictContent = props.containsKey(DICT) &amp;&amp; !Strings.isNullOrEmpty(props.getProperty(DICT)) ? props.getProperty(DICT) : initDefaultDict(); Map&lt;Character, Integer&gt; result = new HashMap&lt;&gt;(dictContent.length(), 1); for (int index = 0; index &lt; dictContent.length(); index++) { result.put(dictContent.charAt(index), index); } return result; } @SneakyThrows private String initDefaultDict() { InputStream inputStream = CharDigestLikeEncryptAlgorithm.class.getClassLoader().getResourceAsStream("algorithm/like/common_chinese_character.dict"); LineProcessor&lt;String&gt; lineProcessor = new LineProcessor&lt;String&gt;() { private final StringBuilder builder = new StringBuilder(); @Override public boolean processLine(final String line) { if (line.startsWith("#") || 0 == line.length()) { return true; } else { builder.append(line); return false; } } @Override public String getResult() { return builder.toString(); } }; return CharStreams.readLines(new InputStreamReader(inputStream, Charsets.UTF_8), lineProcessor); } @Override public String encrypt(final Object plainValue, final EncryptContext encryptContext) { return null == plainValue ? null : digest(String.valueOf(plainValue)); } private String digest(final String plainValue) { StringBuilder result = new StringBuilder(plainValue.length()); for (char each : plainValue.toCharArray()) { char maskedChar = getMaskedChar(each); if ('%' == maskedChar) { result.append(each); } else { result.append(maskedChar); } } return result.toString(); } private char getMaskedChar(final char originalChar) { if ('%' == originalChar) { return originalChar; } if (originalChar &lt;= MAX_NUMERIC_LETTER_CHAR) { return (char) ((originalChar + delta) &amp; mask); } if (charIndexes.containsKey(originalChar)) { return (char) (((charIndexes.get(originalChar) + delta) &amp; mask) + start); } return (char) (((originalChar + delta) &amp; mask) + start); } @Override public String getType() { return "CHAR_DIGEST_LIKE"; } }</code></pre><ul><li>Define the binary <code>mask</code> code to lose precision <code>0b1111_0111_1101</code> (mask).</li> <li>Save common Chinese characters with disrupted order like a <code>map</code> dictionary.</li> <li>Obtain a single string of <code>Unicode</code> for digits, English, and Latin.</li> <li>Obtain an <code>index</code> for a Chinese character belonging to a dictionary.</li> <li>Other characters fetch the <code>Unicode</code> of a single string.</li> <li>Add <code>1 (delta) </code>to the digits obtained by different types above to prevent any original text from appearing in the database.</li> <li>Then convert the offset <code>Unicode</code> into binary, perform the <code>AND</code> operation with <code>mask</code>, and carry out a two-bit digit loss.</li> <li>Directly output digits, English, and Latin after the loss of precision.</li> <li>The remaining characters are converted to decimal and output with the common character <code>start</code> code after the loss of precision.</li> </ul><h2>The fuzzy algorithm development progress</h2> <h3>The first edition</h3> <p>Simply use <code>Unicode</code> and <code>mask</code> code of common characters to perform the <code>AND</code> operation.</p> <pre> <code class="language-plaintext">Mask: 0b11111111111001111101 The original character: 0b1000101110101111讯 After encryption: 0b1000101000101101設</code></pre><p>Assuming we know the key and encryption algorithm, the original string after a backward pass is:</p> <pre> <code class="language-plaintext"> 1.0b1000101100101101 謭 2.0b1000101100101111 謯 3.0b1000101110101101 训 4.0b1000101110101111 讯 5.0b1000101010101101 読 6.0b1000101010101111 誯 7.0b1000101000101111 訯 8.0b1000101000101101 設</code></pre><p>Based on the missing bits, we find that each string can be derived <code>2^n</code> Chinese characters backward. When the <code>Unicode</code> of common Chinese characters is decimal, their intervals are very large. Notice that the Chinese characters inferred backward are not common characters, and it's more likely to infer the original characters.</p> <article class="align-center media media--type-image media--view-mode-default"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="https://opensource.com/sites/default/files/2023-01/41chinesecharacters.jpg" width="832" height="818" alt="Inference of Chinese characters" /></div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>(Xiong Gaoxiang, CC BY-SA 4.0)</p> </div> </article><h3>The second edition</h3> <p>The interval of common Chinese characters in <code>Unicode</code> is irregular. We planned to leave the last few bits of Chinese characters in <code>Unicode</code> and convert them into decimal as an <code>index</code> to fetch some common Chinese characters. This way, when the algorithm is known, uncommon characters won't appear after a backward pass, and distractors are no longer easy to eliminate.</p> <p>If we leave the last few bits of Chinese characters in <code>Unicode</code>, it has something to do with the relationship between the accuracy of fuzzy query and anti-decryption complexity. The higher the accuracy, the lower the decryption difficulty.</p> <p>Let's take a look at the collision degree of common Chinese characters under our algorithm:</p> <p>1. When <code>mask</code>=0b0011_1111_1111:</p> <article class="align-center media media--type-image media--view-mode-default"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="https://opensource.com/sites/default/files/2023-01/42-1-characters.jpg" width="832" height="206" alt="Mask results" /></div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>(Xiong Gaoxiang, CC BY-SA 4.0)</p> </div> </article><p>2. When <code>mask</code>=0b0001_1111_1111:</p> <article class="align-center media media--type-image media--view-mode-default"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="https://opensource.com/sites/default/files/2023-01/42-2-characters.jpg" width="832" height="126" alt="Mask results" /></div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>(Xiong Gaoxiang, CC BY-SA 4.0)</p> </div> </article><p>For the mantissa of Chinese characters, leave 10 and 9 digits. The 10-digit query is more accurate because its collision is much weaker. Nevertheless, if the algorithm and the key are known, the original text of the 1:1 character can be derived backward.</p> <p>The nine-digit query is less accurate because nine-digit collisions are relatively stronger, but there are fewer 1:1 characters. Although we change the collisions regardless of whether we leave ten or nine digits, the distribution is unbalanced due to the irregular <code>Unicode</code> of Chinese characters. The overall collision probability cannot be controlled.</p> <h3>The third edition</h3> <p>In response to the unevenly distributed problem found in the second edition, we take common characters with disrupted order as the dictionary table.</p> <p>1. The encrypted text first looks up the <code>index</code> in the out-of-order dictionary table. We use the <code>index</code> and subscript to replace the <code>Unicode</code> without rules. Use <code>Unicode</code> in case of uncommon characters. (Note: Evenly distribute the code to be calculated as far as possible.)</p> <p>2. The next step is to perform the <code>AND</code> operation with a <code>mask</code> and lose two-bit precision to increase the frequency of collisions.</p> <p>Let's take a look at the collision degree of common Chinese characters under our algorithm:</p> <p>1. When <code>mask</code>=0b1111_1011_1101:</p> <article class="align-center media media--type-image media--view-mode-default"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="https://opensource.com/sites/default/files/2023-01/43-1-characters.png" width="808" height="138" alt="Mask results" /></div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>(Xiong Gaoxiang, CC BY-SA 4.0)</p> </div> </article><p>2. When <code>mask</code>=0b0111_1011_1101:</p> <article class="align-center media media--type-image media--view-mode-default"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="https://opensource.com/sites/default/files/2023-01/43-2-characters.png" width="832" height="120" alt="Mask results" /></div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>(Xiong Gaoxiang, CC BY-SA 4.0)</p> </div> </article><p>When the <code>mask</code> leaves 11 bits, you can see that the collision distribution is concentrated at 1:4. When the <code>mask</code> leaves ten bits, the number becomes 1:8. At this time, we only need to adjust the number of precision losses to control whether the collision is 1:2, 1:4 or 1:8.</p> <p>If the <code>mask</code> is selected as 1, and the algorithm and key are known, there will be a 1:1 Chinese character because we calculate the collision degree of common characters at this time. If we add the missing four bits before the 16-bit binary of Chinese characters, the situation becomes <code>2^5=32</code> cases.</p> <p>Since we encrypt the whole text, even if the individual character is inferred backward, there will be little impact on overall security and will not cause mass data leaks. At the same time, the premise of backward pass is to know the algorithm, key, <code>delta</code>, and dictionary, so it's impossible to achieve from the data in the database.</p> <h2>How to use fuzzy query</h2> <p>Fuzzy query requires the configuration of <code>encryptors</code> (encryption algorithm configuration), <code>likeQueryColumn</code> (fuzzy query column name), and <code>likeQueryEncryptorName</code> (encryption algorithm name of fuzzy query column ) in the encryption configuration.</p> <p>Please refer to the following configuration. Add your own sharding algorithm and data source.</p> <pre> <code class="language-plaintext">dataSources: ds_0: dataSourceClassName: com.zaxxer.hikari.HikariDataSource driverClassName: com.mysql.jdbc.Driver jdbcUrl: jdbc:mysql://127.0.0.1:3306/test?allowPublicKeyRetrieval=true username: root password: root rules: - !ENCRYPT encryptors: like_encryptor: type: CHAR_DIGEST_LIKE aes_encryptor: type: AES props: aes-key-value: 123456abc tables: user: columns: name: cipherColumn: name encryptorName: aes_encryptor assistedQueryColumn: name_ext assistedQueryEncryptorName: aes_encryptor likeQueryColumn: name_like likeQueryEncryptorName: like_encryptor phone: cipherColumn: phone encryptorName: aes_encryptor likeQueryColumn: phone_like likeQueryEncryptorName: like_encryptor queryWithCipherColumn: true props: sql-show: true</code></pre><p>Insert</p> <pre> <code class="language-plaintext">Logic SQL: insert into user ( id, name, phone, sex) values ( 1, '熊高祥', '13012345678', '男') Actual SQL: ds_0 ::: insert into user ( id, name, name_ext, name_like, phone, phone_like, sex) values (1, 'gyVPLyhIzDIZaWDwTl3n4g==', 'gyVPLyhIzDIZaWDwTl3n4g==', '佹堝偀', 'qEmE7xRzW0d7EotlOAt6ww==', '04101454589', '男')</code></pre><p>Update</p> <pre> <code class="language-plaintext">Logic SQL: update user set name = '熊高祥123', sex = '男1' where sex ='男' and phone like '130%' Actual SQL: ds_0 ::: update user set name = 'K22HjufsPPy4rrf4PD046A==', name_ext = 'K22HjufsPPy4rrf4PD046A==', name_like = '佹堝偀014', sex = '男1' where sex ='男' and phone_like like '041%'</code></pre><p>Select</p> <pre> <code class="language-plaintext">Logic SQL: select * from user where (id = 1 or phone = '13012345678') and name like '熊%' Actual SQL: ds_0 ::: select `user`.`id`, `user`.`name` AS `name`, `user`.`sex`, `user`.`phone` AS `phone`, `user`.`create_time` from user where (id = 1 or phone = 'qEmE7xRzW0d7EotlOAt6ww==') and name_like like '佹%'</code></pre><p>Select: federated table sub-query</p> <pre> <code class="language-plaintext">Logic SQL: select * from user LEFT JOIN user_ext on user.id=user_ext.id where user.id in (select id from user where sex = '男' and name like '熊%') Actual SQL: ds_0 ::: select `user`.`id`, `user`.`name` AS `name`, `user`.`sex`, `user`.`phone` AS `phone`, `user`.`create_time`, `user_ext`.`id`, `user_ext`.`address` from user LEFT JOIN user_ext on user.id=user_ext.id where user.id in (select id from user where sex = '男' and name_like like '佹%')</code></pre><p>Delete</p> <pre> <code class="language-plaintext">Logic SQL: delete from user where sex = '男' and name like '熊%' Actual SQL: ds_0 ::: delete from user where sex = '男' and name_like like '佹%'</code></pre><p>The above example demonstrates how fuzzy query columns rewrite SQL in different SQL syntaxes to support fuzzy queries.</p> <h2>Wrap up</h2> <p>This article introduced you to the working principles of fuzzy query and used specific examples to demonstrate how to use it. I hope that through this article, you will have a basic understanding of fuzzy queries.</p> <hr /><p><em>This article was originally published on <a href="https://medium.com/codex/fuzzy-query-for-ciphercolumn-shardingsphere-5-3-0-deep-dive-ad09faea67d3" target="_blank">Medium</a> and has been republished with the author's permission.</em></p> </div> <div class="clearfix text-formatted field field--name-field-article-subhead field--type-text-long field--label-hidden field__item"><p>Learn the working principles of fuzzy queries and follow along with specific examples of how to use them.</p> </div> <div class="field field--name-field-lead-image field--type-entity-reference field--label-hidden field__item"><article class="media media--type-image media--view-mode-caption"> <div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="/sites/default/files/lead-images/code_computer_laptop_hack_work.png" width="1041" height="584" alt="Coding on a computer" title="Coding on a computer" /> </div> </article> </div> <div class="field field--name-field-tags field--type-entity-reference field--label-hidden field__items"> <div class="field__item"><a href="/tags/databases" hreflang="en">Databases</a></div> </div> <div class="hidden field field--name-field-listicle-title field--type-string field--label-hidden field__item">What to read next</div> <div class="field field--name-field-default-license field--type-list-string field--label-hidden field__item"><a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/"> <img alt="Creative Commons License" src="/themes/osdc/assets/img/cc-by-sa-4.png" title="This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License." /></a>This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.</div> <section id='comments' class="field field--name-field-comments field--type-comment field--label-hidden comment-wrapper"> <div class="comments__count"> <div class="login"><a href="/user/register?absolute=1">Register</a> or <a href="/user/login?destination=/feed&amp;absolute=1">Login</a> to post a comment.</div> </div> </section> Wed, 01 Feb 2023 08:00:00 +0000 xionggaoxiang 70427 at https://opensource.com https://opensource.com/article/23/2/fuzzy-query-apache-shardingsphere#comments What's your community thinking pattern? https://opensource.com/article/23/2/community-thinking-pattern <span class="field field--name-title field--type-string field--label-hidden">What&#039;s your community thinking pattern?</span> <span class="field field--name-uid field--type-entity-reference field--label-hidden"><a title="View user profile." href="/users/ron-mcfarland" class="username">Ron McFarland</a></span> <span class="field field--name-created field--type-created field--label-hidden">Wed, 02/01/2023 - 03:00</span> <div class="clearfix text-formatted field field--name-body field--type-text-with-summary field--label-hidden field__item"><p>This article is the second and final part of <a href="https://opensource.com/article/23/1/community-leadership-introducer-chief" target="_blank">a discussion of the research</a> by <a href="https://www.marshall.usc.edu/personnel/david-logan" target="_blank">Dave Logan</a>, <a href="https://www.mindsuitemetrics.com/john-king-profile" target="_blank">Bob King</a>, and <a href="https://drhalee.com/" target="_blank">Halee Fischer-Wright</a>.  If you haven't read the first part yet, you can do so <a href="https://opensource.com/article/23/1/community-leadership-introducer-chief" target="_blank">here</a>. These researchers defined five cultural thinking patterns in communities. In part one, I explained the first three of five thinking patterns. These communities are 20-150 people. I also suggested the responsibilities of an introducer-in-chief. This environmental thinking also refers to how the group behaves and how members talk to each other. To the researchers, each pattern has a identifying perspective:</p> <ul><li>Community thinking pattern #1 (the most negative): "Life is miserable"</li> <li>Community thinking pattern #2: "My life is miserable"</li> <li>Community thinking pattern #3: "I'm great"</li> <li>Community thinking pattern #4: "We're great"</li> <li>Community thinking pattern #5 (the most optimistic): "Life's great"</li> </ul><p>In this article, I continue with their impressions of community thinking pattern #4 and conclude with thinking pattern #5 (the most optimistic).</p> <h2>Community thinking pattern #4: Environment</h2> <p>In this thinking and behaving community pattern, all five <a href="https://theopenorganization.org/definition/open-organization-definition/" target="_blank">open organization principles</a> of transparency, inclusivity, adaptability, collaboration, and community become extremely important to succeed, are actively applied, and are visible. In community thinking pattern #4, a community member transitions from personal gains to direct group or community gains as the focus. As the leader (introducer-in-chief), your role as a direct supervisor is less important than community member delegation. At this stage, you are not as powerful and productive as the total collective community.</p> <p>According to the researchers, the roles of clients, suppliers, friends, mentors, and coaches are blurred at this stage. The people in your community form and focus on value-based relationships and identities. They cluster in larger, more transparent networking groups and develop community pride, as at this level, they focus on common community core values and interdependent strategies.</p> <p>These communities are best formed by assembling like-minded, cooperative individuals who want to advance a specific communal goal. Community members are mostly already at a high individual achieving level, like those in community thinking pattern #3. The community ignores organizational boundaries when looking for community candidates.</p> <p>In the researcher's words, "We're great," and indirectly, "Others are not" is the thought process. These "others" could be even different groups within the same organization.</p> <p>Based on raw talent and high collaboration, this community can be powerful and hard to break. If the wrong (but talented) people are let into this culture, it could be hard to fix. Cooperation and transparency can weaken when the wrong choices are made. During the selection process, there are almost no organizational boundaries. It could explore "on loan" members from other departments, contract or part-time workers, contributors, volunteers or free agents, or anyone dedicated to the community purpose.</p> <p>Regarding collaboration, a community at this stage feels comfortable interacting with other community members only. If invited for a cup of coffee, these community members consider asking at least two other people to come along to get collaboration casually started.</p> <p>When the team hits difficulties, they seek out where solutions might be found. To do that, they seek diverse parties and ideas. They continually review and discuss:</p> <ol><li>What is going well.</li> <li>What is not working well.</li> <li>What the community can do to improve things.</li> </ol><h3>Feelings within early community thinking pattern #4</h3> <p>Only community members who are individually successful in their specialty can move to community thinking pattern #4. If they have not excelled in their field, they come across as weak when they attempt to jump to thinking pattern #4.</p> <p>Moving to community thinking pattern #4 requires group interaction and cooperation, not just individual talent. Therefore, open organization principles are needed, namely:</p> <ul><li>Building communities to achieve more.</li> <li>Inclusivity of others (not just doing things alone or one-on-one giving directives and assignments).</li> <li>Adaptability to a new style of getting things done, collaboration, and transparency.</li> </ul><article class="align-center media media--type-image media--view-mode-default"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="https://opensource.com/sites/default/files/2023-01/outcomes-assets-behaviors.png" width="276" height="284" alt="Outcomes, assets, and behaviors in pattern 4" /></div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>Modified by Ron McFarland from the researcher’s material (Ronald McFarland, CC BY-SA 4.0)</p> </div> </article><p>To advance to a community thinking pattern #4 culture, you must begin with the culture's core values and noble cause (center). These are presumably compelling to everyone in the community, or else they wouldn't join in the first place. Next, move to the outcomes (completing a series of doable, measurable tasks at a predetermined time, leading to a specific goal). Finally, acquire the assets required to accomplish the tasks.</p> <p>These questions come up:</p> <ol><li>What assets do we have right now, and what do we need to achieve the outcome? These could be equipment, technology, land, relationships and network, education, skills, goodwill, brand, public awareness, reputation, culture, or drive/passion.</li> <li>Are there enough assets to achieve what we want?</li> <li>Will the community's behavior accomplish the desired outcome?</li> </ol><p>The leader must consider the community's current culture and thinking to answer the above questions. Considerations include:</p> <ul><li>Who is in the community?</li> <li>What do they care about?</li> <li>What are they working on?</li> <li>What does everyone want?</li> <li>To support members, who can I introduce them to?</li> </ul><p>A big difference between community thinking patterns #3 and #4 is that you, as the leader, must move away from the desire for personal success and redirect your attention to the community's success. This requires you to give up some control and rely on the community members to work with each other directly, make their own decisions, and accepting accountability for their actions. It starts with introducing two members to each other to address a common objective jointly.</p> <article class="align-center media media--type-image media--view-mode-default"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="https://opensource.com/sites/default/files/2023-01/one-on-twoleadership.png" width="394" height="328" alt="One-on-two leadership" /></div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>(Ronald McFarland, CC BY-SA 4.0)</p> </div> </article><p>They start working in gatherings of more than two people (beginning with three, including the leader). Sometimes you introduce two people to each other that might be suitable to work on an issue. You slowly exit the discussion and allow them to take over. Get out of the way and let your team members work directly with one another.</p> <h3>Feelings within middle community thinking pattern #4</h3> <article class="align-center media media--type-image media--view-mode-default"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="https://opensource.com/sites/default/files/2023-01/one-on-eightleadership.png" width="586" height="402" alt="One-on-eight leadership" /></div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>(Ronald McFarland, CC BY-SA 4.0)</p> </div> </article><p>After getting two people working on tasks together, you can start introducing more members with specific functions in mind by continually asking those same questions, starting with "Who is in the community?"</p> <p>These introductions strengthen the community, increase collaboration, and widen inclusivity.</p> <h3>Feelings within late community thinking pattern #4</h3> <article class="align-center media media--type-image media--view-mode-default"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="https://opensource.com/sites/default/files/2023-01/one-on-manyleadership.png" width="552" height="432" alt="One-on-many leadership" /></div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>(Ronald McFarland, CC BY-SA 4.0)</p> </div> </article><p>After members start working with each other on specific tasks that lead to desired outcomes, a high level of inclusivity and collaboration results. This increases community productivity to a globally superior level, leaving all competition behind in their market.</p> <p>After identifying the community's thinking, behavior, and language, your action plan is to explore challenges that your team can't work on without involving outside help. Your goal is to find outside collaboration that encourages community thinking pattern #5 culture. This approach is needed when the community has been so competitive that they need a new, more global challenge.</p> <p>Where once community thinking pattern #4 competed with other communities, now your community is willing to cooperate and collaborate for the overall greater good. This cooperation leads to a more enjoyable working environment. You must encourage and praise your community members every chance you can, and redirects personal praise directed at you to your community members.</p> <h2>Community thinking pattern #5 environment</h2> <p>People with this thinking pattern feel "life is great," according to the researchers. The members of this community have the same pride as thinking pattern #4, but without the competition. Conversely, they hope to collaborate with other communities to achieve a higher calling, like addressing climate change, cancer cures, avoiding war, overcoming disease, or hunger elimination. They seek other communities that have been extremely successful and have similar goals, objectives, and desires.</p> <p>While in community thinking pattern #5, an organization is so successful that it has no need or desire to compete with other communities. Just competing bores them. A leader in community thinking pattern #5 is rewarded by community trust building, hard work, innovation, and collaboration. You are less interested in direct, personal rewards and praise, and more interested in boosting community, inclusivity, and collaboration.</p> <h3>Feelings within community thinking pattern #5</h3> <p>In this community thinking pattern, people are very successful. They're so successful that there is no more competition or desire to compete with others. Instead, they turn more toward vital, global, philanthropic goals. Most financial desires have weakened, because they have accumulated far more assets than they ever would hope to need. They have lost the desire to "win" against others and want to channel energy toward a higher goal.</p> <article class="align-center media media--type-image media--view-mode-default"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="https://opensource.com/sites/default/files/2023-01/pattern5leaders.png" width="562" height="354" alt="Leaders-to-leaders pattern" /></div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>(Ronald McFarland, CC BY-SA 4.0)</p> </div> </article><h2>What community are you in?</h2> <p>In your working environment, think about its greater purpose, and consider membership listening quality, problem-solving quality, ongoing job support, and participation level. Rank those factors from one to four and consider where development is needed. On average, a 3.5 would indicate a competitive thinking pattern #3 community and 4.5 an internally collaborative thinking pattern #4 community. To confirm the group thinking pattern level, also ask yourself what general feeling sticks out among most members:</p> <ul><li>Life is miserable</li> <li>My life is miserable</li> <li>I'm great</li> <li>We're great</li> <li>Life's great</li> </ul><p>Armed with that information, consider what action is required.</p> <h2>Community thinking patterns in action</h2> <p>This is a fictional story of a man named Bob, a man who experienced all community thinking patterns from 1 to 5 (which few of us do).</p> <h3>Community thinking pattern #1 to #2</h3> <p>Imagine that in Bob's early life, he was abandoned by his family. As an orphan, he grew up in an unsafe environment, and had no one to encourage him to achieve anything beyond basic survival.</p> <p>You meet Bob by chance, and after getting to know him, you tell Bob that you might be able to help him out. You are the introducer-in-chief.</p> <p>First, you introduce Bob to Carol, who works as a janitor in a software development company. Carol says she could get Bob a job at this software company, as long as Bob can meet the basic requirements of the position. These are relatively basic requirements: Show up on time, do the work, and so on.</p> <p>While Bob is doing janitorial work in the software company, he asks some of the code writers and programmers about the work they do. David, one of the programmers, is excited by Bob's interest in software development. He asks Bob whether he'd be interested in learning to program. Bob says, "I've always liked to do troubleshooting work and puzzles, so why not?" So David introduces Bob to a wide range of online software development tutorials. David is a secondary introducer-in-chief for Bob.</p> <p>Bob studies a spare on computer there in the company during his spare time. Mentoring him along, David starts to see real talent in Bob, so he introduces Bob to his college professor, Evelyn. Evelyn also sees the talent and drive in Bob, so she counsels him on how to develop his skills further.</p> <h2>Community thinking pattern #2 to #3</h2> <p>Bob proves himself to be a skilled programmer. Over time, his programs become more and more popular. He starts coding for money as a contractor, and as time passes he needs a staff to handle orders coming in. As sales come in, ideas for new programs materialize as well. He has so many ideas that he's overworked. He also hires staff to do programming, but he controls their assignments and tasks very closely. Even though all he's doing is supervising them, he's mentally exhausted at the end of the day.</p> <h2>Community thinking pattern #3 to #4</h2> <p>Bob goes back to Evelyn and asks what to do. Evelyn introduces him to Francis, an open organization consultant for small teams. Francis instructs Bob to get developers to talk to each other more directly, and let them come up with their own solutions and projects. Francis tells Bob that he has to let go of day-to-day supervising operations and start thinking about the overall operation of the team.</p> <p>Over time, just like explained in the book <strong>Team of Teams</strong>, a well-oiled highly-efficient operation is created.</p> <h2>Community thinking pattern #4 to #5</h2> <p>After decades of massive successes, Bob has achieved more than his wildest dreams. His desire to be the best and beat all the competitor weakens. Bob wants to give back in a way that is most globally helpful. He starts approaching people with the same desires and meets Jeff, a global open organization project manager. Jeff specializes in global, multinational, long-term challenges like climate change, global green energy generation, energy waste reduction, food waste reduction, cybersecurity, international relations, world hunger, global migration and many others. Through Jeff's introductions, Bob starts a wide range of multinational projects. Again, just like in the book <strong>Team of Teams</strong>, Bob has gotten global teams collaborating to address really difficult, long-term challenges. Finally, Bob is at peace knowing how much he has achieved and will continue to achieve well into the future.</p> </div> <div class="clearfix text-formatted field field--name-field-article-subhead field--type-text-long field--label-hidden field__item"><p>In the second part of this series, I explore community thinking patterns #4 and #5. Then, I provide a fictional scenario to illustrate thought processes and community influences.</p> </div> <div class="field field--name-field-lead-image field--type-entity-reference field--label-hidden field__item"><article class="media media--type-image media--view-mode-caption"> <div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="/sites/default/files/lead-images/practicing-empathy.jpg" width="1024" height="576" alt="Practicing empathy" title="Practicing empathy" /> </div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>Melissa Hogan, <a href="https://creativecommons.org/licenses/by-sa/4.0" rel="ugc">CC BY-SA 4.0</a>, via Wikimedia Commons</p> </div> </article> </div> <div class="field field--name-field-tags field--type-entity-reference field--label-hidden field__items"> <div class="field__item"><a href="/tags/open-organization" hreflang="en">The Open Organization</a></div> </div> <div class="hidden field field--name-field-listicle-title field--type-string field--label-hidden field__item">What to read next</div> <div class="field field--name-field-default-license field--type-list-string field--label-hidden field__item"><a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/"> <img alt="Creative Commons License" src="/themes/osdc/assets/img/cc-by-sa-4.png" title="This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License." /></a>This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.</div> <section id='comments' class="field field--name-field-comments field--type-comment field--label-hidden comment-wrapper"> <div class="comments__count"> <h2>1 Comment</h2> <div class="login"><a href="/user/register?absolute=1">Register</a> or <a href="/user/login?destination=/feed&amp;absolute=1">Login</a> to post a comment.</div> </div> <article data-comment-user-id="68021" id="comment-220083" class="comment js-comment by-node-author"> <mark class="hidden" data-comment-timestamp="1675298581"></mark> <footer class="comment__meta"> <article class="media media--type-image media--view-mode-byline"> <div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="/sites/default/files/styles/medium/public/osdc_default_avatar_1.png?itok=G0WcUo3c" width="100" height="100" alt="Avatar" title="Avatar" class="image-style-medium" /> </div> </article> <div> <div class="comment__submitted"> <a href="/users/ron-mcfarland">Ron McFarland</a> | February 1, 2023 <div class="rate-widget thumbsup"><form class="comment-content-220083-updown-rate-widget-0 vote-form" id="rate-widget-base-form--2" data-drupal-selector="comment-content-220083-updown-rate-widget-0" action="/feed" method="post" accept-charset="UTF-8"> <div class="rate-widget-thumbs-up"> <div class="thumbsup-rating-wrapper rate-disabled" can-edit="false"><div class="form-item js-form-item form-type-radio js-form-type-radio form-item-value js-form-item-value form-disabled"> <label class="form-item js-form-item form-type-radio js-form-type-radio form-item-value js-form-item-value form-disabled rating-label thumbsup-rating-label thumbsup-rating-label-up vote-pending rate-thumbs-up-btn-up"><input twig-suggestion="rating-input" class="rating-input thumbsup-rating-input form-radio" data-drupal-selector="edit-value-1-2" disabled="disabled" type="radio" id="edit-value-1--2" name="value" value="1" /> </label> </div> </div><input autocomplete="off" data-drupal-selector="form-g-gqc4bzajfp0wiudxjgjf2aasvvn2abqtvae0mfy-y" type="hidden" name="form_build_id" value="form-g_gqC4bZAjfp0WiUdxJgjf2aAsvvn2ABQtvAe0Mfy-Y" /> <input data-drupal-selector="edit-comment-content-220083-updown-rate-widget-0" type="hidden" name="form_id" value="comment_content_220083_updown_rate_widget_0" /> <input class="thumbsup-rating-submit button button--primary js-form-submit form-submit" data-drupal-selector="edit-submit" type="submit" id="edit-submit--4" name="op" value="Save" /> <div class="vote-result js-form-wrapper form-wrapper" data-drupal-selector="edit-result" id="edit-result--2"> <div title="Register or Login to like."><a href="/user/register">Register</a> or <a href="/user/login?destination=/feed">Login</a> to like</div> </div> <div class="rate-score"></div> </div> </form> </div> </div> <div class="content"> <div class="clearfix text-formatted field field--name-comment-body field--type-text-long field--label-hidden field__item"><p>While learning and writing about this "introducer-in-chief" concept, I got to thinking about two types of introductions, one personal and the other virtual. Personally, I live in three physical communities, the Japanese-speaking only world whom I spend well over half more day with, the bilingual Japanese-English world and the English-speaking only world. I have started actively introducing appropriate people to each other, then set back and watch these people interact with each other. I could really feel the power of this activity.</p> <p>Also, there is the virtual world. I'm now reading the book, Homecoming, THE PATH TO PROSPERITY IN A POST-GLOBAL WORLD By Rana Foroohar. She talks about websites that introduce and build local communities. With distributed solar/wind energy, distributed 3D printing, distributed crowd-sourcing and other technologies becoming common, these thinking pattern #4 communities will expand greatly in the years ahead. Keep your eyes open for them in your area. If you are excited about them as I am, you could even start a purpose directed community in your area. Believe me, local community building is rewarding on its own.</p> </div> <drupal-render-placeholder callback="flag.link_builder:build" arguments="0=comment&amp;1=220083&amp;2=report_as_spam" token="Whc2g4KbuedKkQSJMPytrrqNAhm3l6Gr1oTu-z4pJ84"></drupal-render-placeholder><drupal-render-placeholder callback="comment.lazy_builders:renderLinks" arguments="0=220083&amp;1=full&amp;2=en&amp;3=" token="qHmawliEQJUerao5p213s9D7C08JRerP380NILby3D8"></drupal-render-placeholder> </div> </div> </footer> </article> </section> Wed, 01 Feb 2023 08:00:00 +0000 Ron McFarland 70410 at https://opensource.com https://opensource.com/article/23/2/community-thinking-pattern#comments Merge design and code with Penpot https://opensource.com/article/23/1/merge-design-code-penpot <span class="field field--name-title field--type-string field--label-hidden">Merge design and code with Penpot</span> <span class="field field--name-uid field--type-entity-reference field--label-hidden"><a title="View user profile." href="/users/seth" class="username">sethkenlon</a></span> <span class="field field--name-created field--type-created field--label-hidden">Tue, 01/31/2023 - 09:00</span> <div class="clearfix text-formatted field field--name-body field--type-text-with-summary field--label-hidden field__item"><p>For most of the history of computer programming, there's been a gap between the programmers creating an application's code and the designers creating an application's user experience (UX). The two disciplines receive vastly different training, and they use a different set of tools. Programmers use a text editor or an IDE to write code, while designers often draw concepts of widget layout and potential interactions. While some IDEs, like <a href="https://opensource.com/article/20/12/eclipse" target="_blank">Eclipse</a> and <a href="https://opensource.com/article/20/12/netbeans" target="_blank">Netbeans</a>, have interface design components, they're usually focused on widget position and not on widget design. The open source design app <a href="http://penpot.app">Penpot</a> is a collaborative design and prototyping platform. It has a suite of new features that make it easy for designers and developers to work together with familiar workflows. Penpot's design interface lets developers write code in harmony with the design process like no other tool does. And it's come a long way since Opensource.com <a href="https://opensource.com/article/21/9/open-source-design" target="_blank">last looked at it</a>. Its latest features don't just improve your experience with Penpot, they propel the open source Penpot app past similar and proprietary tools.</p> <h2 id="_prototyping">Prototyping with Penpot</h2> <p>One of the common problems with trying to design how an application might work best is that, at the time of designing, the application doesn't exist yet. A designer can visualize and storyboard to help both the design team and the programmer understand what to aim for. But it's a process that requires iteration and feedback as developers start to implement UX concepts, and designs change to react to the reality of code.</p> <p>With Penpot, you can create a "working" prototype of your web or mobile application. You can connect buttons with specific actions, triggering changes in layout based on user input. And this can all be done before any code for the project exists.</p> <p>The most important aspect of this isn't the ability to do a mock-up, though. Everything done in Penpot for an app's design has usable layout data that developers can use in the final project. Penpot isn't just a great drawing and layout tool. It informs the coding process.</p> <p>Rather than providing just a visual list of designer-specific elements, like properties, colors, and typography, Penpot now integrates code output directly into the design workspace (like developer tools in a web browser). Designers and developers share the same space for design and front-end development, getting specifications in whatever format they need.</p> <article class="align-center media media--type-image media--view-mode-default"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="https://opensource.com/sites/default/files/2022-07/Current%20Penpot%20interface.png" width="1670" height="932" alt="Image of the current Penpot interface" /></div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>(Andrey Antukh, CC BY-SA 4.0)</p> </div> </article><h2 id="_memory_unlock">Memory unlock</h2> <p>Many online design tools use proprietary technology to provide some fancy features, but at the price of essentially becoming an application, you don't run so much as access through a browser. Penpot uses open web standards, though, and is rendered by your web browser. That means Penpot has access to the web browser's maximum available memory, which makes Penpot the first online prototype and layout application with design scalability. You can provide more options, more mock-ups, and more pitches. Plus, you can open your design space to more concurrent collaborators with no fear of running out of application memory.</p> <p></p><div class="embedded-resource-list callout-float-right"> <div class="field field--name-title field--type-string field--label-hidden field__item">Our favorite resources about open source</div> <div class="field field--name-links field--type-link field--label-hidden field__items"> <div class="field__item"><a href="https://opensource.com/downloads/cheat-sheet-git?intcmp=7016000000127cYAAQ">Git cheat sheet</a></div> <div class="field__item"><a href="https://developers.redhat.com/cheat-sheets/advanced-linux-commands/?intcmp=7016000000127cYAAQ">Advanced Linux commands cheat sheet</a></div> <div class="field__item"><a href="https://opensource.com/tags/alternatives?intcmp=7016000000127cYAAQ">Open source alternatives</a></div> <div class="field__item"><a href="https://www.redhat.com/en/services/training/rh024-red-hat-linux-technical-overview?intcmp=7016000000127cYAAQ">Free online course: RHEL technical overview</a></div> <div class="field__item"><a href="https://opensource.com/downloads/cheat-sheets?intcmp=7016000000127cYAAQ">Check out more cheat sheets</a></div> </div> </div> <h2 id="_self_hosting_and_saas">Self-hosting and SaaS</h2> <p>Penpot is open source, so you don't have to use it on the cloud if that doesn't fit your workflow. You can self-host Penpot easily in a container, using it as a local application on your own workstation or hosting it for your organization on your own server.</p> <h2 id="_open_source_design">Open source design</h2> <p>I've written an <a href="https://opensource.com/article/21/12/open-source-design-penpot">introductory article to Penpot</a> previously, and since then the application has only gotten better. If you're looking to bring coders and stakeholders into your design process, then give Penpot a try.</p> </div> <div class="clearfix text-formatted field field--name-field-article-subhead field--type-text-long field--label-hidden field__item"><p>Bridge the gap between programming and design with Penpot, an open source design workspace.</p> </div> <div class="field field--name-field-lead-image field--type-entity-reference field--label-hidden field__item"><article class="media media--type-image media--view-mode-caption"> <div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="/sites/default/files/lead-images/painting_computer_screen_art_design_creative.png" width="1040" height="585" alt="Painting art on a computer screen" title="Painting art on a computer screen" /> </div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>Opensource.com</p> </div> </article> </div> <div class="field field--name-field-tags field--type-entity-reference field--label-hidden field__items"> <div class="field__item"><a href="/tags/art-and-design" hreflang="en">Art and design</a></div> <div class="field__item"><a href="/tags/open-studio" hreflang="en">Open Studio</a></div> <div class="field__item"><a href="/tags/alternatives" hreflang="en">Alternatives</a></div> </div> <div class="hidden field field--name-field-listicle-title field--type-string field--label-hidden field__item">What to read next</div> <div class="field field--name-field-default-license field--type-list-string field--label-hidden field__item"><a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/"> <img alt="Creative Commons License" src="/themes/osdc/assets/img/cc-by-sa-4.png" title="This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License." /></a>This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.</div> <section id='comments' class="field field--name-field-comments field--type-comment field--label-hidden comment-wrapper"> <div class="comments__count"> <div class="login"><a href="/user/register?absolute=1">Register</a> or <a href="/user/login?destination=/feed&amp;absolute=1">Login</a> to post a comment.</div> </div> </section> Tue, 31 Jan 2023 14:00:00 +0000 sethkenlon 70443 at https://opensource.com https://opensource.com/article/23/1/merge-design-code-penpot#comments Use Terraform to manage an OpenStack cluster https://opensource.com/article/23/1/terraform-manage-openstack-cluster <span class="field field--name-title field--type-string field--label-hidden">Use Terraform to manage an OpenStack cluster</span> <span class="field field--name-uid field--type-entity-reference field--label-hidden"><a title="View user profile." href="/users/ajscanlas" class="username">ajscanlas</a></span> <span class="field field--name-created field--type-created field--label-hidden">Tue, 01/31/2023 - 03:00</span> <div class="clearfix text-formatted field field--name-body field--type-text-with-summary field--label-hidden field__item"><p>After having an OpenStack production and home lab for a while, I can definitively say that provisioning a workload and managing it from an Admin and Tenant perspective is important.</p> <p>Terraform is an open source Infrastructure-as-Code (IaC) software tool used for provisioning networks, servers, cloud platforms, and more. Terraform is a declarative language that can act as a blueprint of the infrastructure you're working on. You can manage it with Git, and it has a strong <a href="https://opensource.com/article/21/3/gitops" target="_blank">GitOps</a> use case.</p> <p>This article covers the basics of managing an OpenStack cluster using Terraform. I recreate the OpenStack Demo project using Terraform.</p> <h2>Install Terraform</h2> <p>I use CentOS as a jump host, where I run Terraform. Based on the official documentation, the first step is to add the Hashicorp repository:</p> <pre> <code class="language-shell">$ sudo dnf config-manager \ --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo</code></pre><p>Next, install Terraform:</p> <pre> <code class="language-shell">$ sudo dnf install terraform -y</code></pre><p>Verify the installation:</p> <pre> <code class="language-shell">$ terraform –version</code></pre><p>If you see a version number in return, you have installed Terraform.</p> <p></p><div class="embedded-resource-list callout-float-right"> <div class="field field--name-title field--type-string field--label-hidden field__item">Explore the open source cloud</div> <div class="field field--name-links field--type-link field--label-hidden field__items"> <div class="field__item"><a href="https://www.redhat.com/en/services/training/do092-developing-cloud-native-applications-microservices-architectures?intcmp=7013a000002gp8aAAA">Free online course: Developing cloud-native applications with microservices</a></div> <div class="field__item"><a href="https://www.redhat.com/en/engage/managed-cloud-services-s-202202210207?intcmp=7013a000002gp8aAAA">eBook: Modernize your IT with managed cloud services</a></div> <div class="field__item"><a href="https://cloud.redhat.com/products/dedicated/?intcmp=7013a000002gp8aAAA">Try for 60 days: Red Hat OpenShift Dedicated</a></div> <div class="field__item"><a href="https://www.redhat.com/en/services/training/do080-deploying-containerized-applications-technical-overview?intcmp=7013a000002gp8aAAA">Free online course: Containers, Kubernetes and Red Hat OpenShift</a></div> <div class="field__item"><a href="https://www.redhat.com/en/topics/containers/what-is-kubernetes?intcmp=7013a000002gp8aAAA">What is Kubernetes?</a></div> <div class="field__item"><a href="https://www.redhat.com/en/topics/edge-computing?intcmp=7013a000002gp8aAAA">Understanding edge computing</a></div> <div class="field__item"><a href="https://www.redhat.com/architect/?intcmp=7013a000002gp8aAAA">Latest articles for IT architects</a></div> </div> </div> <h2>Create a Terraform script for the OpenStack provider</h2> <p>In Terraform, you need a provider. A provider is a converter that Terraform calls to convert your <code>.tf</code> into API calls to the platform you are orchestrating.</p> <p>There are three types of providers: Official, Partner, and Community:</p> <ul><li>Official providers are Hashicorp maintained.</li> <li>Partner providers are maintained by technology companies that partner with Hashicorp.</li> <li>Community providers are maintained by open source community members.</li> </ul><p>There is a good Community provider for OpenStack in this <a href="https://registry.terraform.io/providers/terraform-provider-openstack/openstack/1.49.0" target="_blank">link</a>. To use this provider, create a <code>.tf</code> file and call it <code>main.tf</code>.</p> <pre> <code class="language-shell">$ vi main.tf</code></pre><p>Add the following content to <code>main.tf</code>:</p> <pre> <code class="language-text">terraform { required_version = "&gt;= 0.14.0" required_providers { openstack = { source = "terraform-provider-openstack/openstack" version = "1.49.0" } } } provider "openstack" { user_name = “OS_USERNAME” tenant_name = “OS_TENANT” password = “OS_PASSWORD” auth_url = “OS_AUTH_URL” region = “OS_REGION” }</code></pre><p>You need to change the <strong>OS_USERNAME</strong>, <strong>OS_TENANT</strong>, <strong>OS_PASSWORD</strong>, <strong>OS_AUTH_URL</strong>, and <strong>OS_REGION</strong> variables for it to work.</p> <h2>Create an Admin Terraform file</h2> <p>OpenStack Admin files focus on provisioning external networks, routers, users, images, tenant profiles, and quotas.</p> <p>This example provisions flavors, a router connected to an external network, a test image, a tenant profile, and a user.</p> <p>First, create an <code>AdminTF</code> directory for the provisioning resources:</p> <pre> <code class="language-shell">$ mkdir AdminTF $ cd AdminTF</code></pre><p>In the <code>main.tf</code>, add the following:</p> <pre> <code class="language-text">terraform { required_version = "&gt;= 0.14.0" required_providers { openstack = { source = "terraform-provider-openstack/openstack" version = "1.49.0" } } } provider "openstack" { user_name = “OS_USERNAME” tenant_name = “admin” password = “OS_PASSWORD” auth_url = “OS_AUTH_URL” region = “OS_REGION” } resource "openstack_compute_flavor_v2" "small-flavor" { name = "small" ram = "4096" vcpus = "1" disk = "0" flavor_id = "1" is_public = "true" } resource "openstack_compute_flavor_v2" "medium-flavor" { name = "medium" ram = "8192" vcpus = "2" disk = "0" flavor_id = "2" is_public = "true" } resource "openstack_compute_flavor_v2" "large-flavor" { name = "large" ram = "16384" vcpus = "4" disk = "0" flavor_id = "3" is_public = "true" } resource "openstack_compute_flavor_v2" "xlarge-flavor" { name = "xlarge" ram = "32768" vcpus = "8" disk = "0" flavor_id = "4" is_public = "true" } resource "openstack_networking_network_v2" "external-network" { name = "external-network" admin_state_up = "true" external = "true" segments { network_type = "flat" physical_network = "physnet1" } } resource "openstack_networking_subnet_v2" "external-subnet" { name = "external-subnet" network_id = openstack_networking_network_v2.external-network.id cidr = "10.0.0.0/8" gateway_ip = "10.0.0.1" dns_nameservers = ["10.0.0.254", "10.0.0.253"] allocation_pool { start = "10.0.0.1" end = "10.0.254.254" } } resource "openstack_networking_router_v2" "external-router" { name = "external-router" admin_state_up = true external_network_id = openstack_networking_network_v2.external-network.id } resource "openstack_images_image_v2" "cirros" { name = "cirros" image_source_url = "https://download.cirros-cloud.net/0.6.1/cirros-0.6.1-x86_64-disk.img" container_format = "bare" disk_format = "qcow2" properties = { key = "value" } } resource "openstack_identity_project_v3" "demo-project" { name = "Demo" } resource "openstack_identity_user_v3" "demo-user" { name = "demo-user" default_project_id = openstack_identity_project_v3.demo-project.id password = "demo" }</code></pre><h2>Create a Tenant Terraform file</h2> <p>As a Tenant, you usually create VMs. You also create network and security groups for the VMs.</p> <p>This example uses the user created above by the Admin file.</p> <p>First, create a <code>TenantTF</code> directory for Tenant-related provisioning:</p> <pre> <code class="language-shell">$ mkdir TenantTF $ cd TenantTF</code></pre><p>In the <code>main.tf</code>, add the following:</p> <pre> <code class="language-text">terraform { required_version = "&gt;= 0.14.0" required_providers { openstack = { source = "terraform-provider-openstack/openstack" version = "1.49.0" } } } provider "openstack" { user_name = “demo-user” tenant_name = “demo” password = “demo” auth_url = “OS_AUTH_URL” region = “OS_REGION” } resource "openstack_compute_keypair_v2" "demo-keypair" { name = "demo-key" public_key = "ssh-rsa ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ" } resource "openstack_networking_network_v2" "demo-network" { name = "demo-network" admin_state_up = "true" } resource "openstack_networking_subnet_v2" "demo-subnet" { network_id = openstack_networking_network_v2.demo-network.id name = "demo-subnet" cidr = "192.168.26.0/24" } resource "openstack_networking_router_interface_v2" "demo-router-interface" { router_id = “XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX” subnet_id = openstack_networking_subnet_v2.demo-subnet.id } resource "openstack_compute_instance_v2" "demo-instance" { name = "demo" image_id = "YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY" flavor_id = "3" key_pair = "demo-key" security_groups = ["default"] metadata = { this = "that" } network { name = "demo-network" } }</code></pre><h2>Initialize your Terraform</h2> <p>After creating the Terraform files, you need to initialize Terraform.</p> <p>For Admin:</p> <pre> <code class="language-shell">$ cd AdminTF $ terraform init $ terraform fmt</code></pre><p>For Tenants:</p> <pre> <code class="language-shell">$ cd TenantTF $ terraform init $ terraform fmt</code></pre><p>Command explanation:</p> <ul><li><code>terraform init</code> downloads the provider from the registry to use in provisioning this project.</li> <li><code>terraform fmt</code> formats the files for use in repositories.</li> </ul><h2>Create a Terraform plan</h2> <p>Next, create a plan for you to see what resources will be created.</p> <p>For Admin:</p> <pre> <code class="language-shell">$ cd AdminTF $ terraform validate $ terraform plan</code></pre><p>For Tenants:</p> <pre> <code class="language-shell">$ cd TenantTF $ terraform validate $ terraform plan</code></pre><p>Command explanation:</p> <ul><li><code>terraform validate</code> validates whether the <code>.tf</code> syntax is correct.</li> <li><code>terraform plan</code> creates a plan file in the cache where all managed resources can be tracked in creation and destroy.</li> </ul><h2>Apply your first TF</h2> <p>To deploy the resources, use the <code>terraform apply</code> command. This command applies all resource states in the plan file.</p> <p>For Admin:</p> <pre> <code class="language-shell">$ cd AdminTF $ terraform apply</code></pre><p>For Tenants:</p> <pre> <code class="language-shell">$ cd TenantTF $ terraform apply</code></pre><h2>Next steps</h2> <p>Previously, I wrote an <a href="https://opensource.com/article/20/12/openstack-raspberry-pi" target="_blank">article</a> on deploying a minimal OpenStack cluster on a Raspberry Pi. You can discover how to have more detailed <a href="https://www.ansible.com/blog/ansible-vs.-terraform-demystified?intcmp=7013a000002qLH8AAM" target="_blank">Terraform and Ansible</a> configurations and implement some CI/CD with GitLab.</p> </div> <div class="clearfix text-formatted field field--name-field-article-subhead field--type-text-long field--label-hidden field__item"><p>Terraform is a declarative language that can act as a blueprint of the infrastructure you&#039;re working on.</p> </div> <div class="field field--name-field-lead-image field--type-entity-reference field--label-hidden field__item"><article class="media media--type-image media--view-mode-caption"> <div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="/sites/default/files/lead-images/rh_003601_05_mech_osyearbook2016_cloud_cc.png" width="520" height="292" alt="What&#039;s new in OpenStack in 2016: A look at the Newton release" title="What&#039;s new in OpenStack in 2016: A look at the Newton release" /> </div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>Opensource.com</p> </div> </article> </div> <div class="field field--name-field-tags field--type-entity-reference field--label-hidden field__items"> <div class="field__item"><a href="/tags/cloud" hreflang="en">Cloud</a></div> <div class="field__item"><a href="/tags/openstack" hreflang="en">OpenStack</a></div> <div class="field__item"><a href="/tags/automation" hreflang="en">Automation</a></div> <div class="field__item"><a href="/tags/cicd" hreflang="en">CI/CD</a></div> </div> <div class="hidden field field--name-field-listicle-title field--type-string field--label-hidden field__item">What to read next</div> <div class="field field--name-field-default-license field--type-list-string field--label-hidden field__item"><a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/"> <img alt="Creative Commons License" src="/themes/osdc/assets/img/cc-by-sa-4.png" title="This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License." /></a>This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.</div> <section id='comments' class="field field--name-field-comments field--type-comment field--label-hidden comment-wrapper"> <div class="comments__count"> <div class="login"><a href="/user/register?absolute=1">Register</a> or <a href="/user/login?destination=/feed&amp;absolute=1">Login</a> to post a comment.</div> </div> </section> Tue, 31 Jan 2023 08:00:00 +0000 ajscanlas 70429 at https://opensource.com https://opensource.com/article/23/1/terraform-manage-openstack-cluster#comments Automate Mastodon interactions with Python https://opensource.com/article/23/1/mastodon-api-python <span class="field field--name-title field--type-string field--label-hidden">Automate Mastodon interactions with Python</span> <span class="field field--name-uid field--type-entity-reference field--label-hidden"><a title="View user profile." href="/users/moshez" class="username">Moshe Zadka</a></span> <span class="field field--name-created field--type-created field--label-hidden">Tue, 01/31/2023 - 03:00</span> <div class="clearfix text-formatted field field--name-body field--type-text-with-summary field--label-hidden field__item"><p>The federated <a href="https://opensource.com/article/22/11/switch-twitter-mastodon" target="_blank">Mastodon</a> social network has gotten very popular lately. It's fun to post on social media, but it's also fun to automate your interactions. There is some <a href="https://docs.joinmastodon.org/client/intro/" target="_blank">documentation</a> of the client-facing API, but it's a bit light on examples. This article aims to help with that.</p> <p>You should be fairly confident with Python before trying to follow along with this article. If you're not comfortable in Python yet, check out Seth Kenlon's <a href="https://opensource.com/article/17/10/python-101" target="_blank">Getting started with Python</a> article and my <a href="https://opensource.com/article/20/12/learn-python" target="_blank">Program a simple game</a> article.</p> <p></p><div class="embedded-resource-list callout-float-right"> <div class="field field--name-title field--type-string field--label-hidden field__item">More Python resources</div> <div class="field field--name-links field--type-link field--label-hidden field__items"> <div class="field__item"><a href="https://www.redhat.com/en/topics/middleware/what-is-ide?intcmp=7016000000127cYAAQ">What is an IDE?</a></div> <div class="field__item"><a href="https://opensource.com/downloads/cheat-sheet-python-37-beginners?intcmp=7016000000127cYAAQ">Cheat sheet: Python 3.7 for beginners</a></div> <div class="field__item"><a href="https://opensource.com/resources/python/gui-frameworks?intcmp=7016000000127cYAAQ">Top Python GUI frameworks</a></div> <div class="field__item"><a href="https://opensource.com/downloads/7-essential-pypi-libraries?intcmp=7016000000127cYAAQ">Download: 7 essential PyPI libraries</a></div> <div class="field__item"><a href="https://developers.redhat.com/?intcmp=7016000000127cYAAQ">Red Hat Developers</a></div> <div class="field__item"><a href="https://opensource.com/tags/python?intcmp=7016000000127cYAAQ">Latest Python articles</a></div> </div> </div> <h2>Create an application</h2> <p>The first step is to go to <strong>Preferences</strong> in Mastodon and select the <strong>Development</strong> category. In the <strong>Development</strong> panel, click on the <strong>New Applications</strong> button.</p> <p>After creating an application, copy the access token. Be careful with it. This is your authorization to post to your Mastodon account.</p> <p>There are a few Python modules that can help.</p> <ul><li>The <a href="https://opensource.com/article/22/3/python-httpx" target="_blank">httpx</a> module is useful, given that it is a web API.</li> <li>The <code>getpass</code> module allows you to paste the token into the session safely.</li> <li>Mastodon uses HTML as its post content, so a nice way to display HTML inline is useful.</li> <li>Communication is all about timing, and the <code>dateutil</code> and <code>zoneinfo</code> modules will help deal with that.</li> </ul><p>Here's what my typical import list looks like:</p> <pre> <code class="language-python">import httpx import getpass from IPython.core import display from dateutil import parser import zoneinfo</code></pre><p>Paste in the token into the <code>getpass</code> input:</p> <pre> <code class="language-python">token=getpass.getpass()</code></pre><p>Create the <code>httpx.Client</code>:</p> <pre> <code class="language-python">client = httpx.Client(headers=dict(Authorization=f"Bearer {token}"))</code></pre><p>The <strong>verify_credentials</strong> method exists to verify that the token works. It's a good test, and it gives you useful metadata about your account:</p> <pre> <code class="language-python">res = client.get("https://mastodon.social/api/v1/accounts/verify_credentials")</code></pre><p>You can query your Mastodon identity:</p> <pre> <code class="language-python">res.raise_for_status() result = res.json() result["id"], result["username"] &gt;&gt;&gt; ('27639', 'moshez')</code></pre><p>Your mileage will vary, but you get your internal ID and username in response. The ID can be useful later.</p> <p>For now, abstract away the <strong>raise_for_status</strong> and parse the JSON output:</p> <pre> <code class="language-python">def parse(result):     result.raise_for_status()     return result.json()</code></pre><p>Here's how this can be useful. Now you can check your account data by ID. This is a nice way to cross-check consistency:</p> <pre> <code class="language-python">result = parse(client.get("https://mastodon.social/api/v1/accounts/27639")) result["username"] &gt;&gt;&gt; 'moshez'</code></pre><p>But the interesting thing, of course, is to get your timeline. Luckily, there's an API for that:</p> <pre> <code class="language-python">statuses = parse(client.get("https://mastodon.social/api/v1/timelines/home")) len(statuses) &gt;&gt;&gt; 20</code></pre><p>It's just a count of posts, but that's enough for now. There's no need to deal with paging just yet. The question is, what can you do with a list of your posts? Well, you can query it for all kinds of interesting data. For instance, who posted the fourth status?</p> <pre> <code class="language-python">some_status = statuses[3] some_status["account"]["username"] &gt;&gt;&gt; 'donwatkins'</code></pre><p>Wonderful, a tweet from fellow Opensource.com correspondent <a href="https://opensource.com/users/don-watkins" target="_blank">Don Watkins</a>! Always great content. I'll check it out:</p> <pre> <code class="language-python">display.HTML(some_status["content"]) &lt;p&gt;Just finished installed &lt;span class="h-card"&gt;&lt;a href="https://fosstodon.org/@fedora" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank"&gt;@&lt;span&gt;fedora&lt;/span&gt;&lt;/a&gt;&lt;/span&gt; &lt;a href="https://fosstodon.org/tags/Silverblue" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank"&gt;#&lt;span&gt;Silverblue&lt;/span&gt;&lt;/a&gt; 37 on &lt;span class="h-card"&gt;&lt;a href="https://fosstodon.org/@system76" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank"&gt;@&lt;span&gt;system76&lt;/span&gt;&lt;/a&gt;&lt;/span&gt; &lt;a href="https://fosstodon.org/tags/DarterPro" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank"&gt;#&lt;span&gt;DarterPro&lt;/span&gt;&lt;/a&gt;&lt;/p&gt;</code></pre><p>"Just" finished? Wait, when was this tweet posted? I live in California, so I want the time in my local zone:</p> <pre> <code class="language-python">california = zoneinfo.ZoneInfo("US/Pacific") when = parser.isoparse(some_status["created_at"]) print(when.astimezone(california)) &gt;&gt;&gt; 2022-12-29 13:56:56-08:00</code></pre><p>Today (at the time of this writing), a little before 2 PM. Talk about timeliness.</p> <p>Do you want to see the post for yourself? Here's the URL:</p> <pre> <code class="language-python">some_status["url"] &gt;&gt;&gt; 'https://fosstodon.org/@donwatkins/109599196234639478'</code></pre><p>Enjoy tooting, now with 20% more API!</p> </div> <div class="clearfix text-formatted field field--name-field-article-subhead field--type-text-long field--label-hidden field__item"><p>Follow along as I play with the Mastodon API to create a new application.</p> </div> <div class="field field--name-field-lead-image field--type-entity-reference field--label-hidden field__item"><article class="media media--type-image media--view-mode-caption"> <div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="/sites/default/files/lead-images/OSDC_women_computing_5.png" width="520" height="292" alt="Women in computing and open source v5" title="Women in computing and open source v5" /> </div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>kris krüg</p> </div> </article> </div> <div class="field field--name-field-tags field--type-entity-reference field--label-hidden field__items"> <div class="field__item"><a href="/tags/python" hreflang="en">Python</a></div> <div class="field__item"><a href="/tags/alternatives" hreflang="en">Alternatives</a></div> </div> <div class="hidden field field--name-field-listicle-title field--type-string field--label-hidden field__item">What to read next</div> <div class="field field--name-field-default-license field--type-list-string field--label-hidden field__item"><a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/"> <img alt="Creative Commons License" src="/themes/osdc/assets/img/cc-by-sa-4.png" title="This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License." /></a>This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.</div> <section id='comments' class="field field--name-field-comments field--type-comment field--label-hidden comment-wrapper"> <div class="comments__count"> <h2>2 Comments</h2> <div class="login"><a href="/user/register?absolute=1">Register</a> or <a href="/user/login?destination=/feed&amp;absolute=1">Login</a> to post a comment.</div> </div> <article data-comment-user-id="15542" id="comment-220079" class="comment js-comment"> <mark class="hidden" data-comment-timestamp="1675175148"></mark> <footer class="comment__meta"> <article class="media media--type-image media--view-mode-byline"> <div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="/sites/default/files/styles/medium/public/osdc_default_avatar_1.png?itok=G0WcUo3c" width="100" height="100" alt="Avatar" title="Avatar" class="image-style-medium" /> </div> </article> <div> <div class="comment__submitted"> <a href="/users/don-watkins">Don Watkins</a> | January 31, 2023 <div class="rate-widget thumbsup"><form class="comment-content-220079-updown-rate-widget-0 vote-form" id="rate-widget-base-form--3" data-drupal-selector="comment-content-220079-updown-rate-widget-0" action="/feed" method="post" accept-charset="UTF-8"> <div class="rate-widget-thumbs-up"> <div class="thumbsup-rating-wrapper rate-disabled" can-edit="false"><div class="form-item js-form-item form-type-radio js-form-type-radio form-item-value js-form-item-value form-disabled"> <label class="form-item js-form-item form-type-radio js-form-type-radio form-item-value js-form-item-value form-disabled rating-label thumbsup-rating-label thumbsup-rating-label-up vote-pending rate-thumbs-up-btn-up"><input twig-suggestion="rating-input" class="rating-input thumbsup-rating-input form-radio" data-drupal-selector="edit-value-1-3" disabled="disabled" type="radio" id="edit-value-1--3" name="value" value="1" /> </label> </div> </div><input autocomplete="off" data-drupal-selector="form-omjaspvgkjk4es6w72tpv9uhyoht6ta8kuspoqcaoqe" type="hidden" name="form_build_id" value="form-omjAsPvGKjk4ES6W72Tpv9UhYoHt6tA8KuspOqCaoQE" /> <input data-drupal-selector="edit-comment-content-220079-updown-rate-widget-0" type="hidden" name="form_id" value="comment_content_220079_updown_rate_widget_0" /> <input class="thumbsup-rating-submit button button--primary js-form-submit form-submit" data-drupal-selector="edit-submit" type="submit" id="edit-submit--6" name="op" value="Save" /> <div class="vote-result js-form-wrapper form-wrapper" data-drupal-selector="edit-result" id="edit-result--3"> <div title="Register or Login to like."><a href="/user/register">Register</a> or <a href="/user/login?destination=/feed">Login</a> to like</div> </div> <div class="rate-score"></div> </div> </form> </div> </div> <div class="content"> <div class="clearfix text-formatted field field--name-comment-body field--type-text-long field--label-hidden field__item"><p>Thank you for the mention Moshez. That was very kind of you. I really like this article and you've inspired me to try this use of Python. </p> </div> <drupal-render-placeholder callback="flag.link_builder:build" arguments="0=comment&amp;1=220079&amp;2=report_as_spam" token="HTKjeKdNiLENHQE4l-6t17FKBvNzOB7FLGaaBkw4Mvo"></drupal-render-placeholder><drupal-render-placeholder callback="comment.lazy_builders:renderLinks" arguments="0=220079&amp;1=full&amp;2=en&amp;3=" token="SMp4Sflf1B37YrSwkNdV-vpEN9bhJ8iWSyBGdkLwVA4"></drupal-render-placeholder> </div> </div> </footer> </article> <article data-comment-user-id="520361" id="comment-220080" class="comment js-comment"> <mark class="hidden" data-comment-timestamp="1675221991"></mark> <footer class="comment__meta"> <article class="media media--type-image media--view-mode-byline"> <div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="/sites/default/files/styles/medium/public/osdc_default_avatar_1.png?itok=G0WcUo3c" width="100" height="100" alt="Avatar" title="Avatar" class="image-style-medium" /> </div> </article> <div> <div class="comment__submitted"> <a href="/users/pdecker">pdecker</a> | January 31, 2023 <div class="rate-widget thumbsup"><form class="comment-content-220080-updown-rate-widget-0 vote-form" id="rate-widget-base-form--4" data-drupal-selector="comment-content-220080-updown-rate-widget-0" action="/feed" method="post" accept-charset="UTF-8"> <div class="rate-widget-thumbs-up"> <div class="thumbsup-rating-wrapper rate-disabled" can-edit="false"><div class="form-item js-form-item form-type-radio js-form-type-radio form-item-value js-form-item-value form-disabled"> <label class="form-item js-form-item form-type-radio js-form-type-radio form-item-value js-form-item-value form-disabled rating-label thumbsup-rating-label thumbsup-rating-label-up vote-pending rate-thumbs-up-btn-up"><input twig-suggestion="rating-input" class="rating-input thumbsup-rating-input form-radio" data-drupal-selector="edit-value-1-4" disabled="disabled" type="radio" id="edit-value-1--4" name="value" value="1" /> </label> </div> </div><input autocomplete="off" data-drupal-selector="form-4a9rihgad-qeunrqcocwcsmh1vql8yxdqbujticiwie" type="hidden" name="form_build_id" value="form-4a9RiHgAd-qeUNrqcOcwcSmH1vQl8YXdQbUJtICIWiE" /> <input data-drupal-selector="edit-comment-content-220080-updown-rate-widget-0" type="hidden" name="form_id" value="comment_content_220080_updown_rate_widget_0" /> <input class="thumbsup-rating-submit button button--primary js-form-submit form-submit" data-drupal-selector="edit-submit" type="submit" id="edit-submit--8" name="op" value="Save" /> <div class="vote-result js-form-wrapper form-wrapper" data-drupal-selector="edit-result" id="edit-result--4"> <div title="Register or Login to like."><a href="/user/register">Register</a> or <a href="/user/login?destination=/feed">Login</a> to like</div> </div> <div class="rate-score"></div> </div> </form> </div> </div> <div class="content"> <div class="clearfix text-formatted field field--name-comment-body field--type-text-long field--label-hidden field__item"><p>Very nice. Another reason for me to get started with Mastodon.</p> </div> <drupal-render-placeholder callback="flag.link_builder:build" arguments="0=comment&amp;1=220080&amp;2=report_as_spam" token="luk-x0z_szFxPHtUsn5jVnuy3tsp-H6Ji38ORCU0hDU"></drupal-render-placeholder><drupal-render-placeholder callback="comment.lazy_builders:renderLinks" arguments="0=220080&amp;1=full&amp;2=en&amp;3=" token="fTVIEIREEY02dlilcuRCoMFZokwM3PZoXeoV3qdtIyM"></drupal-render-placeholder> </div> </div> </footer> </article> </section> Tue, 31 Jan 2023 08:00:00 +0000 Moshe Zadka 70423 at https://opensource.com https://opensource.com/article/23/1/mastodon-api-python#comments How to use GitOps to automate Terraform https://opensource.com/article/23/1/automate-terraform-gitops <span class="field field--name-title field--type-string field--label-hidden">How to use GitOps to automate Terraform</span> <span class="field field--name-uid field--type-entity-reference field--label-hidden"><a title="View user profile." href="/users/robstr" class="username">robstr</a></span> <span class="field field--name-created field--type-created field--label-hidden">Mon, 01/30/2023 - 03:00</span> <div class="clearfix text-formatted field field--name-body field--type-text-with-summary field--label-hidden field__item"><p>GitOps as a workflow is perfect for application delivery, mostly used in <a href="https://www.redhat.com/en/topics/containers/what-is-kubernetes?intcmp=7013a000002qLH8AAM" target="_blank">Kubernetes</a> environments, but it is also possible to use for infrastructure. In a typical GitOps scenario, you might want to look at solutions like Crossplane as a Kubernetes-native alternative, while most traditional infrastructure are still used with CI/CD pipelines. There are several benefits of creating your deployment platform with Kubernetes as the base, but it also means that more people would have to have that particular skill set. One of the benefits of an Infrastructure-as-Code tool like Terraform is that it is easy to learn, and doesn't require much specialized knowledge.</p> <p>When my team was building our platform services, we wanted everyone to be able to contribute. Most, if not all, of our engineers use Terraform on a daily basis. They know how to create Terraform modules that can be used in several scenarios and for several customers. While there are several ways of automating Terraform, we would like to utilize a proper GitOps workflow as much as possible.</p> <h2 id="how-does-the-terraform-controller-work">How does the Terraform controller work</h2> <p>While searching for alternatives for running Terraform using Kubernetes, I found several controllers and operators, but none that I felt had as much potential as the <a href="https://github.com/weaveworks/tf-controller/" rel="noopener" target="_blank">tf-controller from Weaveworks</a>. We are already using Flux as our GitOps tool. The tf-controller works by utilizing some of the core functionality from Flux, and has a custom resource for Terraform deployments. The source controller takes care of fetching our modules, the Kustomize controllers apply the Terraform resources, and then the controller spins up static pods (called runners) that run your Terraform commands.</p> <p>The Terraform resource looks something like this:</p> <pre> <code class="language-yaml">apiVersion: infra.contrib.fluxcd.io/v1alpha1 kind: Terraform metadata: name: helloworld namespace: flux-system spec: interval: 1m approvePlan: auto path: ./terraform/module sourceRef: kind: GitRepository name: helloworld namespace: flux-system</code></pre><p>There are a few things to note on the specs here. The interval in the spec controls how often the controller starts up the runner pods. This then performs a <code>terraform plan</code> on your root module, which is defined by the path parameter.</p> <p>This particular resource is set to automatically approve a plan. This means that if there is a difference between the plan and the current state of the target system, a new runner will run to apply the changes automatically. This makes the process as "GitOps" as possible, but you can disable this. If you disable it, you have to manually approve plans. You can do this either by using the Terraform Controller CLI or by updating your manifests with a reference to the commit which should be applied. For more details, see the <a href="https://docs.gitops.weave.works/docs/terraform/Using%20Terraform%20CRD/provision/#manually-apply-resources" rel="noopener" target="_blank">documentation</a> on manual approval.</p> <p>The tf-controller utilizes the source controller from Flux. The <code>sourceRef</code> attribute is used to define which source resource you want to use, just like a Flux Kustomization resource would.</p> <h2 id="advanced-deployments">Advanced deployments</h2> <p>While the example above works, it's not the type of deployment my team would normally do. When not defining a backend storage, the state would get stored in the cluster, which is fine for testing and development. But for production, I prefer that the state file is stored somewhere outside the cluster. I don't want this defined in the root module directly, as I want to reuse our root modules in several deployments. This means I have to define our backend in our Terraform resource.</p> <p>Here is an example of how I set up custom backend configurations. You can find all available backends in the <a href="https://developer.hashicorp.com/terraform/language/settings/backends/configuration" rel="noopener" target="_blank">Terraform docs</a>:</p> <pre> <code class="language-yaml">apiVersion: infra.contrib.fluxcd.io/v1alpha1 kind: Terraform metadata: name: helloworld namespace: flux-system spec: backendConfig: customConfiguration: | backend "azurerm" { resource_group_name = "rg-terraform-mgmt" storage_account_name = "stgextfstate" container_name = "tfstate" key = "helloworld.tfstate" } ...</code></pre><p>Storing the state file outside the cluster means that I can redeploy our cluster. But then there is no storage dependency. There is no need for backup or state migration. As soon as the new cluster is up, it runs the commands against the same state, and I am back in business.</p> <p>Another advanced move is dependencies between modules. Sometimes we design deployments like a two-stage rocket, where one deployment sets up certain resources that the next one uses. In these scenarios, we need to make sure that our Terraform is written in such a fashion so that we output any data needed as inputs for the second module, and ensure that the first module has a successful run first.</p> <p>These two examples are from code used while demonstrating dependencies, and all the code can be found on my <a href="https://github.com/roberthstrand/gitops-terraform/tree/main" rel="noopener" target="_blank">GitHub</a>. Some of the code is omitted for brevity's sake:</p> <pre> <code class="language-yaml">apiVersion: infra.contrib.fluxcd.io/v1alpha1 kind: Terraform metadata: name: shared-resources namespace: flux-system spec: ... writeOutputsToSecret: name: shared-resources-output ...</code></pre><pre> <code class="language-yaml">apiVersion: infra.contrib.fluxcd.io/v1alpha1 kind: Terraform metadata: name: workload01 namespace: flux-system spec: ... dependsOn: - name: shared-resources ... varsFrom: - kind: Secret name: shared-resources-output ...</code></pre><p>In the deployment that I call <strong>shared-resources</strong>, you see that I defined a secret where the outputs from the deployment should be stored. In this case, the outputs are the following:</p> <pre> <code class="language-bash">output "subnet_id" { value = azurerm_virtual_network.base.subnet.*.id[0] } output "resource_group_name" { value = azurerm_resource_group.base.name }</code></pre><p>In the <strong>workload01</strong> deployment, I first define our dependency with the <code>dependsOn</code> attribute, which makes sure that <strong>shared-resources</strong> has a successful run before scheduling <strong>workload01</strong>. The outputs from <strong>shared-resources</strong> is then used as inputs in <strong>workload01</strong>, which is the reason why I want to wait.</p> <p></p><div class="embedded-resource-list callout-float-right"> <div class="field field--name-title field--type-string field--label-hidden field__item">More on automation</div> <div class="field field--name-links field--type-link field--label-hidden field__items"> <div class="field__item"><a href="https://www.redhat.com/en/engage/automated-enterprise-ebook-20171115?intcmp=701f2000000h4RcAAI">Download now: The automated enterprise eBook</a></div> <div class="field__item"><a href="https://www.redhat.com/en/services/training/do007-ansible-essentials-simplicity-automation-technical-overview?intcmp=701f2000000h4RcAAI">Free online course: Ansible essentials</a></div> <div class="field__item"><a href="https://opensource.com/downloads/ansible-cheat-sheet?intcmp=701f2000000h4RcAAI">Ansible cheat sheet</a></div> <div class="field__item"><a href="https://opensource.com/downloads/home-automation-ebook?intcmp=701f2000000h4RcAAI">eBook: A practical guide to home automation using open source tools</a></div> <div class="field__item"><a href="https://opensource.com/downloads/ansible-quickstart?intcmp=701f2000000h4RcAAI">A quickstart guide to Ansible</a></div> <div class="field__item"><a href="https://www.redhat.com/en/engage/automation-at-edge-20220727?intcmp=701f2000000h4RcAAI">eBook: 7 examples of automation on the edge</a></div> <div class="field__item"><a href="https://opensource.com/tags/automation?intcmp=701f2000000h4RcAAI">More articles about open source automation</a></div> </div> </div> <h2 id="why-the-controller-pattern-and-not-pipelines-or-terraform-cloud">Why not pipelines or Terraform Cloud</h2> <p>The most common approach to automating Terraform is either by using CI/CD pipelines or Terraform Cloud. Using pipelines for Terraform works fine, but usually ends up with needing to copy pipeline definitions over and over again. There are solutions to that, but by using the tf-controller you have a much more declarative approach to defining what you want your deployments to look like rather than defining the steps in an imperative fashion.</p> <p>Terraform Cloud has introduced a lot of features that overlap with using the GitOps workflow, but using the tf-controller does not exclude you from using Terraform Cloud. You could use Terraform Cloud as the backend for your deployment, only automating the runs through the tf-controller.</p> <p>The reason that my team uses this approach is that we already deploy applications using GitOps, and we have much more flexibility as to how we can offer these capabilities as a service. We can control our implementation through APIs, making self-service more accessible to both our operators and end-users. The details around our platform approach are such a big topic, that we will have to return to those in its own article.</p> <hr /><p><em>This article was originally published on the <a href="https://blog.amestofortytwo.com/automate-terraform-gitops-flux/" target="_blank">author's blog</a> and has been republished with permission.</em></p> </div> <div class="clearfix text-formatted field field--name-field-article-subhead field--type-text-long field--label-hidden field__item"><p>Instead of using CI/CD pipelines or Terraform Cloud, try this alternative approach to automating Terraform using Flux and GitOps.</p> </div> <div class="field field--name-field-lead-image field--type-entity-reference field--label-hidden field__item"><article class="media media--type-image media--view-mode-caption"> <div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="/sites/default/files/lead-images/gears_devops_learn_troubleshooting_lightbulb_tips_520.png" width="520" height="292" alt="Tips and gears turning" title="Tips and gears turning" /> </div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>opensource.com</p> </div> </article> </div> <div class="field field--name-field-tags field--type-entity-reference field--label-hidden field__items"> <div class="field__item"><a href="/tags/cicd" hreflang="en">CI/CD</a></div> <div class="field__item"><a href="/tags/automation" hreflang="en">Automation</a></div> <div class="field__item"><a href="/tags/kubernetes" hreflang="en">Kubernetes</a></div> </div> <div class="hidden field field--name-field-listicle-title field--type-string field--label-hidden field__item">What to read next</div> <div class="field field--name-field-default-license field--type-list-string field--label-hidden field__item"><a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/"> <img alt="Creative Commons License" src="/themes/osdc/assets/img/cc-by-sa-4.png" title="This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License." /></a>This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.</div> <section id='comments' class="field field--name-field-comments field--type-comment field--label-hidden comment-wrapper"> <div class="comments__count"> <div class="login"><a href="/user/register?absolute=1">Register</a> or <a href="/user/login?destination=/feed&amp;absolute=1">Login</a> to post a comment.</div> </div> </section> Mon, 30 Jan 2023 08:00:00 +0000 robstr 70430 at https://opensource.com Learn to code a simple game in Zig https://opensource.com/article/23/1/learn-zig-programming <span class="field field--name-title field--type-string field--label-hidden">Learn to code a simple game in Zig</span> <span class="field field--name-uid field--type-entity-reference field--label-hidden"><a title="View user profile." href="/users/moshez" class="username">Moshe Zadka</a></span> <span class="field field--name-created field--type-created field--label-hidden">Mon, 01/30/2023 - 03:00</span> <div class="clearfix text-formatted field field--name-body field--type-text-with-summary field--label-hidden field__item"><p id="learn-zig-by-coding-a-game">Writing the same application in multiple languages is a great way to learn new ways to program. Most programming languages have certain things in common, such as:</p> <ul><li>Variables</li> <li>Expressions</li> <li>Statements</li> </ul><p>These concepts are the basis of most programming languages. Once you understand them, you can take the time you need to figure out the rest.</p> <p>Furthermore, programming languages usually share some similarities. Once you know one programming language, you can learn the basics of another by recognizing its differences.</p> <p>A good tool for learning a new language is by practicing with a standard program.<br /> This allows you to focus on the language, not the program's logic. I'm doing that in this article series using a "guess the number" program, in which the computer picks a number between 1 and 100 and asks you to guess it. The program loops until you guess the number correctly.</p> <p>This program exercises several concepts in programming languages:</p> <ul><li>Variables</li> <li>Input</li> <li>Output</li> <li>Conditional evaluation</li> <li>Loops</li> </ul><p>It's a great practical experiment to learn a new programming language.</p> <h2 id="guess-the-number-in-zig-basic">Guess the number in Zig basic</h2> <p>Zig is still in the alpha stage, and subject to change. This article is correct as of zig version 0.11. Install <code>zig</code> by going to the <a href="https://ziglang.org/download/">downloads</a> directory and downloading the appropriate version for your operating system and architecture:</p> <pre> <code class="language-bash">const std = @import("std"); fn ask_user() !i64 { const stdin = std.io.getStdIn().reader(); const stdout = std.io.getStdOut().writer(); var buf: [10]u8 = undefined; try stdout.print("Guess a number between 1 and 100: ", .{}); if (try stdin.readUntilDelimiterOrEof(buf[0..], '\n')) |user_input| { return std.fmt.parseInt(i64, user_input, 10); } else { return error.InvalidParam; } } pub fn main() !void { const stdout = std.io.getStdOut().writer(); var prng = std.rand.DefaultPrng.init(blk: { var seed: u64 = undefined; try std.os.getrandom(std.mem.asBytes(&amp;seed)); break :blk seed; }); const value = prng.random().intRangeAtMost(i64, 1, 100); while (true) { const guess = try ask_user(); if (guess == value) { break; } const message = if (guess &lt; value) "low" else "high"; try stdout.print("Too {s}\n", .{message}); } try stdout.print("That's right\n", .{}); }</code></pre><p>The first line <code>const std = @import("std");</code> imports the Zig <a href="https://ziglang.org/documentation/master/std/" rel="noopener" target="_blank">standard library</a>.<br /> Almost all programs will need it.</p> <h2 id="the-ask_user-function-in-zig">The ask_user function in Zig</h2> <p>The function <code>ask_user()</code> returns a 64-bit integer <em>or</em> an error. This is what the <code>!</code> (exclamation mark) notes. This means if there is an I/O issue or the user enters an invalid input, the function returns an error.</p> <p>The <code>try</code> operator calls a function and return its value. If it returns an error, it immediately returns from the <em>calling</em> function with an error. This allows explicit, but easy, error propagation. The first two lines in <code>ask_user</code> alias contains some constants from <code>std</code>.<br /> This makes the following I/O code simpler.</p> <p>This line prints the prompt:</p> <pre> <code class="language-bash">try stdout.print("Guess a number between 1 and 100: ", .{});</code></pre><p>It automatically returns a failure if the print fails (for example, writing to a closed terminal).</p> <p>This line defines the buffer into which user input is read:</p> <pre> <code class="language-bash">var buf: [10]u8 = undefined;</code></pre><p>The expression inside the if clause reads user input into the buffer:</p> <pre> <code class="language-bash">(try stdin.readUntilDelimiterOrEof(buf[0..], '\n')) |user_input|</code></pre><p>The expression returns the slice of the buffer that was read into. This is assigned to the variable <code>user_input</code>, which is only valid inside the <code>if</code> block.</p> <p>The function <code>std.fmt.parseInt</code> returns an error if the number cannot be parsed.<br /> This error is propagated to the caller. If no bytes have been read, the function immediately returns an error.</p> <p></p><div class="embedded-resource-list callout-float-right"> <div class="field field--name-title field--type-string field--label-hidden field__item">Programming and development</div> <div class="field field--name-links field--type-link field--label-hidden field__items"> <div class="field__item"><a href="https://developers.redhat.com/?intcmp=7016000000127cYAAQ">Red Hat Developers Blog</a></div> <div class="field__item"><a href="https://opensource.com/downloads/cheat-sheets?intcmp=7016000000127cYAAQ">Programming cheat sheets</a></div> <div class="field__item"><a href="https://www.redhat.com/en/services/training/learning-subscription?intcmp=7016000000127cYAAQ">Try for free: Red Hat Learning Subscription</a></div> <div class="field__item"><a href="https://opensource.com/downloads/bash-programming-guide?intcmp=7016000000127cYAAQ">eBook: An introduction to programming with Bash</a></div> <div class="field__item"><a href="https://developers.redhat.com/cheat-sheets/bash-shell-cheat-sheet?intcmp=7016000000127cYAAQ">Bash shell scripting cheat sheet</a></div> <div class="field__item"><a href="https://developers.redhat.com/e-books/modernizing-enterprise-java?intcmp=7016000000127cYAAQ">eBook: Modernizing Enterprise Java</a></div> <div class="field__item"><a href="https://opensource.com/downloads/building-applications?intcmp=7016000000127cYAAQ">An open source developer's guide to building applications</a></div> </div> </div> <h2 id="the-main-function">The main function</h2> <p>The function begins by getting a random number. It uses <code>std.rand.DefaultPrng</code>.</p> <p>The function initializes the random number generator with <code>std.os.getrandom</code>. It then uses the generator to get a number in the range of <code>1</code> to <code>100</code>.</p> <p>The <code>while</code> loop continues while <code>true</code> is true, which is forever. The only way out is with the <code>break</code>, which happens when the guess is equal to the random value.</p> <p>When the <code>guess</code> is not equal, the <code>if</code> statement returns the string <code>low</code> or <code>high</code> depending on the guess. This is interpolated into the message to the user.</p> <p>Note that <code>main</code> is defined as <code>!void</code>, which means it can also return an error. This allows using the <code>try</code> operator inside <code>main</code>.</p> <h2 id="sample-output">Sample output</h2> <p>An example run, after putting the program in <code>main.zig</code>:</p> <pre> <code class="language-bash">$ zig run main.zig Guess a number between 1 and 100: 50 Too high Guess a number between 1 and 100: 25 Too low Guess a number between 1 and 100: 37 Too low Guess a number between 1 and 100: 42 Too high Guess a number between 1 and 100: 40 That's right</code></pre><h2 id="summary">Summary</h2> <p>This "guess the number" game is a great introductory program for learning a new programming language because it exercises several common programming concepts in a pretty straightforward way. By implementing this simple game in different programming languages, you can demonstrate some core concepts of the languages and compare their details.</p> <p>Do you have a favorite programming language? How would you write the "guess the number" game in it? Follow this article series to see examples of other programming languages that might interest you!</p> </div> <div class="clearfix text-formatted field field--name-field-article-subhead field--type-text-long field--label-hidden field__item"><p>Practice programming in Zig by writing a &quot;guess the number&quot; game.</p> </div> <div class="field field--name-field-lead-image field--type-entity-reference field--label-hidden field__item"><article class="media media--type-image media--view-mode-caption"> <div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <img loading="lazy" src="/sites/default/files/lead-images/OSDC_women_computing_2.png" width="520" height="292" alt="Woman sitting in front of her computer" title="Women in computing and open source" /> </div> <div class="field field--name-field-caption field--type-text-long field--label-hidden caption field__item"><span class="caption__byline">Image by: </span><p>Ray Smith</p> </div> </article> </div> <div class="field field--name-field-tags field--type-entity-reference field--label-hidden field__items"> <div class="field__item"><a href="/tags/programming" hreflang="en">Programming</a></div> </div> <div class="field field--name-field-listicle-title field--type-string field--label-hidden field__item">What to read next</div> <div class="field field--name-field-listicles field--type-entity-reference field--label-hidden field__items"> <div class="field__item"><a href="/article/23/2/learn-basic-coding-game" hreflang="en">Learn Basic by coding a game</a></div> </div> <div class="field field--name-field-default-license field--type-list-string field--label-hidden field__item"><a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/"> <img alt="Creative Commons License" src="/themes/osdc/assets/img/cc-by-sa-4.png" title="This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License." /></a>This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.</div> <section id='comments' class="field field--name-field-comments field--type-comment field--label-hidden comment-wrapper"> <div class="comments__count"> <div class="login"><a href="/user/register?absolute=1">Register</a> or <a href="/user/login?destination=/feed&amp;absolute=1">Login</a> to post a comment.</div> </div> </section> Mon, 30 Jan 2023 08:00:00 +0000 Moshe Zadka 70422 at https://opensource.com https://opensource.com/article/23/1/learn-zig-programming#comments