Explaining to someone without a technical background can be challenging. That challenge is compounded by the creative names developers give to their products. If we are not careful, people may get confused when terms like Go, React, or Flutter slide into our conversations without explanations. The fact that we have so many software products around, sometimes solving the same issue in a slightly different way, has always raised a question in my mind. We can see how Occam's razor can be applied to benefit developers who bear the cost of deciding and picking the right tool for the right job. Why do we continue to have more software doing similar things and add to the complexity?

Of course, I can always defer the argument by citing that things exist for a reason. Bearing in mind the why, the sharing I attended (on Cross Platform Application Development) last night was insightful on the practical considerations when selecting a desirable framework for the job. I experienced a similar conundrum when I was settling on a framework for mobile development. Similar choices between React Native and Flutter. My answer back then: Flutter. I will explain some of my considerations (or the lack thereof) with what was mentioned by the speakers.

First Look​

As a student, I like the fact that we do not always have to consider aspects of software durability or robustness when choosing something to learn or use in a side project. When picking a new tool or framework, the first thing that I would do is to take stock of what tools I already know. Though not entirely accurate, that would have given me a rough idea of how well-received and how much support for that technology online. One thing that I learned from the talk is to look at more data points such as Github stars and issue counts. As an example, React-native has 97.5k stars, and Flutter has 129k stars. The popularity of Flutter was one reason why I decided to take it up. Another aspect of popularity is the issue count, which I am still ambivalent about what it portrays. React-native has over 1k issues when Futter has over 5k issues. While I understand that more users might mean more complaints, other factors might also affect this metric. Whether the maintainers of these frameworks actively listen and patch the software to fix pertinent issues can be crucial when the software plays a huge role as a framework that drives the entire code execution.

Familiar VS Trendy​

While the above point explains why I would prefer a trendy framework, I also understand the importance of familiarity (language or otherwise). It is especially true when development velocity and product quality are critical. A constant theme mentioned by the speakers was that developers would like to iterate on their products, frequent and fast. Therefore, choosing a familiar tool that most developers are aware of, will make onboarding and upskilling less a hassle. To illustrate, I only got to know Dart as a language used when working with Flutter and my experience with it is only within the period of mobile development that I have done. During the live coding session last night, the presenter was facing a bug due to the displacement of a variable. Looking at the code, I thought that it might be due to the incorrect assignment of the variable, which was not the case! It was the result of not me being unfamiliar with the tool used. Especially in frameworks with a lot of conventions AND configurations, developers might have to slow down significantly. Choosing something that we get exposed to frequently is a fair bet that we can adopt it fast and have it work for us in a reasonable amount of time.

The Right Job​

Mobile applications have an unfair advantage of wider outreach and convenience. The ability for mobile frameworks to work cross-platform is, similarly, the biggest selling point that they have against native toolchains. As the sole developer or part of a small team, the right job often includes maximizing output and reducing maintenance costs. To have one framework that builds for both IOS and Android is a bargain that we cannot ignore. To relate to my example, I chose Flutter also because even if I started developing with the sole focus on IOS, it would be a breeze to account for the minor differences so that it works well on Android. An interesting point that I learned from the Shopee example is that sometimes tools can work together to generate greater yield. I have never thought of situations like isolating the support service into web pages(because of the significant number of custom service personnel that use it). If we can put more emphasis on our target audience and to find ways best benefit them, the user experience and the developer experience can both increase dramatically.

It is fascinating that we say to pick the RIGHT tool for the RIGHT job when there is no clear definition for what is RIGHT. Worse still, what is RIGHT is going to change in no time. My conclusion: do research, try things out, adapt and change when necessary.

P.S. This is repost of my writing assignment for CS3216.

Unlike solving problems in an exam paper, building a product starts with defining a problem worth solving. Problems can come from pain points that we endure going through our quotidian life. Or, they can be astute observations of obstacles that people around us encounter. Some issues might seem mundane, while others have half-decent, duct tape solutions(a better analogy nowadays is a combination of Excel, Google Docs, and Whatsapp). I will be elaborating on three lessons I have learned in the recent UIUX workshop, in terms of product ideation and validation.

The first key lesson that I learned from the workshop (on Beauty is more than skin deep) is to be wary of bad ideas. We want to build something that stands out from existing competitions. We want mass adoption from human beings who have an innate tendency to prefer stability over change. If the above is true, we need to be able to answer the following: Is the problem that we are describing real? Does the proposed solution solve it? It's easy to see the relevance of the first question. During the workshop, we had a group proposing a product idea to help students find mentors easily. Critical response from the audience was how that product would provide utility to people who avail themselves as mentors. Assuming that the product does help their target users(students), it lacked the consideration of another significant group of involved parties(potential mentors). To go deeper into the above example, the proposed solution might not effectively address an inherent issue at hand: perhaps there's a lack of motivated mentors?

My second learning point is on making a product that offers better solutions. I would like to share an anecdote that resonates with the content of the workshop. I was reading blog posts by Kent C. Dodds (a software engineer and OSS educator) and came across his open-source project named "split-guide". It is a tool to help generate code for his workshops. Quoting the problem and the proposed solution highlighted in the repository's Readme for context:

Problem: Managing workshop repositories. A great way to do a workshop repo is to have exercises and exercises-final directories. The problem is keeping these two directories in sync. Solution: This allows you to create a template for both of these in a templates directory. Based on special text in these files, you can specify what parts of the file you want in exercises and exercises-final. This allows you to co-locate what you're going to be showing workshop attendees and what the final answer should be.

Having done many coding tutorials, I can see why managing workshop repositories can be a pain. However, it turned out that "split-guide" is no longer maintained. In the latest Pull Request (https://github.com/kentcdodds/split-guide/pull/24), Kent remarked that he thought the solution was neat but complex. The better solution/alternative that Kent settled for? "copy/paste/modify". This is such an apt illustration of how a solution that has more friction can go south.

The last point is on the importance of user validation. A surprising fact that I got out of the UX critique session is how much we can gain from iterations with a simple design and a group of target users poking at it. Feedback from users can guide us in making design choices that meet the eye and lead users down the intended happy path. One of the more striking comments that I remembered from the session was that users might not appreciate the sophisticated algorithms underneath if they are not motivated to use the product.

P.S. This is repost of my writing assignment for CS3216.

Motivation​

I was looking through the drafts that I wrote and thought that this one could be salvaged. I have done some simple graph visualization projects and I still think they are fun to work on. Though most of the time we just learn the APIs of the graphing libraries of our choice, these libraries work wonders to present data. So here is a short walk-through of how I would use HighCharts to showcase data from the Dev.to API. As an example, Dev.to API is used to retrieve details of 1000 articles to plot them in the form of a "packedbubble" graph. The size of each bubble refers to the reaction count (positive_reaction_count + comment_count). Then when hovered over, the title, URL, and the count of the article will be shown. The articles that have over 1000 reaction counts will be labeled. I have also arbitrarily chosen to only display articles of 8 categories/tags (More details in Step 2).

Initial Preparation​

I have come to realize that a part of programming work is converting data from one form to the other. To use a front-end graphing library, in my experience having data in JSON format is the most convenient. However, there are times when the data source could be coming from CSV or Excel spreadsheet. We could either write some conversion scripts in Python or have some preprocessing steps in JavaScript. Papa Parse is one such JS helper package that I have previously used. Even if we have APIs that return us JSON formatted data, we might still need to manipulate it into the format that the charting library expects.

In this working example, I am choosing Highcharts for their rich features and extremely good documentations. They have many Jsfiddle examples that could serve as a good reference/starting point. However, do note that paid license is required to use their products commercially. To use it for free, note the following:

Non-profit organisations, schools and personal websites can enjoy our software for free under a Creative Commons (CC) Attribution-Non-Commercial license. In order to obtain a non-commercial license, please fill out this form.

The first thing to do is to find out what structure of the data is expected by Hightcharts. Sometimes this information can be confusing to figure out, given that documentations of graph/chart libraries are filled with options and explanations. So, we look at examples. This is one such example I found browsing their documentation. Looking at the code, it is easy to identify that data to be used in the chart is specified here:

series: [{
        data: [1, 4, 3, 5],
        type: 'column',
        name: 'Fruits'
    }]

So a series contains an array of individual groups of data. The actual data points are within the attribute data, in the form of an array. Upon further inspection of other examples, we can see that the data points need not be primitives like numbers or strings. They could be objects containing the data point and its metadata such as its name or other attributes. Now we are ready to proceed.

Step 1:​

Fetch 1000 articles from Dev.to using the API:

async function makeGetRequestAndReturnJson() {
    const response = await fetch('https://dev.to/api/articles?per_page=1000');
    return await response.json();
}

Step 2:​

Manipulate the data into the required format. Each individual data point is of the following format:

{
  'title': 'someTitle',
  'url': 'someUrl',
  'value': 'someReactionCount'
}

And the code to filter and consolidate the data is as follows (I might have gone too functional in the data processing part, use of for-loops is possible too) :

async function processData() {
    const targetTags = ['react', 'opensource', 'codenewbie', 'beginners', 'tutorial', 'webdev', 'showdev', 'productivity'];
    const seriesData = [{
        name: 'react',
        data: []
    },
    {
        name: 'opensource',
        data: []
    },
    {
        name: 'codenewbie',
        data: []
    },
    {
        name: 'beginners',
        data: []
    },
    {
        name: 'tutorial',
        data: []
    },
    {
        name: 'webdev',
        data: []
    },

    {
        name: 'showdev',
        data: []
    },
    {
        name: 'productivity',
        data: []
    }];
    const data = await makeGetRequestAndReturnJson();
    const filteredData = data.filter(article => article.tag_list.some(tag => targetTags.includes(tag)))
    filteredData.forEach(article => {
        const filteredTags = article.tag_list.filter(tag => targetTags.includes(tag))
        filteredTags.forEach(tag => {
            seriesData.find(type => type.name === tag).data.push(
                {
                    title: article.title,
                    url: article.url,
                    value: article.comments_count + article.positive_reactions_count
                })
        });
    })
    return seriesData;
}

Step 3:​

Setup and use the prepared data in the graph configuration process:

async function setupGraph() {
    const seriesData = await processData()
    chart = new Highcharts.chart('container', {
        chart: {
            type: 'packedbubble',
            height: '50%',
        },
        title: {
            text: 'Visualizing Dev.to articles'
        },
        tooltip: {
            useHTML: true,
            stickOnContact: true,
            pointFormat: '<b>{point.title}:</b> <br/>Reaction Count: {point.value} <br/><a target="_blank" href={point.url}>{point.url}</a>'
        },

        plotOptions: {
            packedbubble: {
                useSimulation: false, // true for a better animation
                minSize: '30%',
                maxSize: '100%',
                zMin: 0,
                zMax: 2000, // the max value of the bubble
                layoutAlgorithm: {
                    gravitationalConstant: 0.01,
                    splitSeries: false, 
                    seriesInteraction: true,
                    dragBetweenSeries: true,
                    parentNodeLimit: true,
                },
                dataLabels: {
                    enabled: true,
                    format: '{point.title}',
                    filter: {
                        property: 'y',
                        operator: '>',
                        value: 1000 // labeling the articles with over 1000 in positive reaction counts
                    },
                    style: {
                        // adjusting of styles for data labels
                        color: 'black',
                        // textOutline: 'none',
                        // fontWeight: 'normal',
                    },

                },
            }
        },
        series: seriesData,
    });
}

Step 4:​

Invoke the function call when ready:

// trigger setupGraph function on document ready
document.addEventListener('DOMContentLoaded', () => {
    setupGraph();
})

Step 5:​

Create a basic HTML page to run the script and display the outcome:

<!DOCTYPE html>
<html lang="en">
  <head>
    <title>DevTo Visualization</title>
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <meta charset="utf-8" />
    <!-- Load jQuery -->
    <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
    <script src="https://code.highcharts.com/highcharts.js"></script>
    <script src="https://code.highcharts.com/highcharts-more.js"></script>
    <script src="https://code.highcharts.com/modules/exporting.js"></script>
    <script src="https://code.highcharts.com/modules/accessibility.js"></script>
  </head>
  <body>
    <div id="container"></div>
    <script src="index.js"></script>
  </body>
</html>

Conclusion​

Putting everything together, here is the link to see the visualization in action. Here is the link to the GitHub repo if you are interested in the code.

In terms of difficulty, most of the complexity lies in knowing the settings and configurations of the library in use. I think the harder part is finding out what to visualize and the appropriate graph/chart type to use. What story should the data tell? In my quick example, I guess it shows that people really enjoy "collectible" and "mark for further usage" kinds of articles 😂.

Some further extension ideas:

• Explore the Dev.to API to get some interesting data, such as
  • filter the tags using the API parameter to only retrieve articles of certain tags
  • Retrieve articles that you authored
• Explore other graph/chart types available