Collecting Data from Chess.com: A Look at Writing Your Own Custom Telegraf Plugin
By
Community /
Use Cases, Product, Developer
Jul 26, 2021
Navigate to:
This article was written by InfluxData Summer 2021 interns Noé Garcia, Mya Longmire, Aidan Tai, Dane Strandboge and Merrick Tian.
Telegraf is a powerful tool that enables you to gather metrics and information from stacks, sensors or systems. Telegraf handles the heavy lifting and computation allowing you to spend more time focusing on what you want to do with your newly collected data. For this reason, our team decided to build a simple plugin for Telegraf that collects information and user metrics from Chess.com. We use the official Chess.com published data API to gather information from. In this blog post, we will go over how we built this plugin in an attempt to guide the reader through the process so they can build their own plugins.
Getting started
To begin our plugin, we forked and downloaded the Telegraf project from its official GitHub repository. All Telegraf plugins are located within the plugins directory, and since we are gathering information from an external source for the system, we placed our source code within the plugins/inputs directory. There, we create a directory called chess to hold all of our code. The working path to our source directory appears as follows: telegraf/plugins/inputs/chess.
The documentation provides a sample Telegraf configuration to build our plugin off of. The following is the sample configuration given that we can place within our chess.go file within our plugin directory:
package simple
// simple.go
import (
"github.com/influxdata/telegraf"
"github.com/influxdata/telegraf/plugins/inputs"
)
type Simple struct {
Ok bool `toml:"ok"`
Log telegraf.Logger `toml:"-"`
}
func (s *Simple) Description() string {
return "a demo plugin"
}
func (s *Simple) SampleConfig() string {
return `
## Indicate if everything is fine
ok = true
`
}
// Init is for setup, and validating config.
func (s *Simple) Init() error {
return nil
}
func (s *Simple) Gather(acc telegraf.Accumulator) error {
if s.Ok {
acc.AddFields("state", map[string]interface{}{"value": "pretty good"}, nil)
} else {
acc.AddFields("state", map[string]interface{}{"value": "not great"}, nil)
}
return nil
}
func init() {
inputs.Add("simple", func() telegraf.Input { return &Simple{} })
}After placing the sample code within our Go file, we can start to alter the code to fit our plugin. First, we need to update our package name to the name of our plugin by writing package chess. Next, we need to change the data structure below the imports statement to represent our plugin configuration and match the README. Finally, we can update any other names or data types that we have altered in our code. With that, we end up with the following:
package chess
// chess.go
import (
"github.com/influxdata/telegraf"
"github.com/influxdata/telegraf/plugins/inputs"
)
type Chess struct {
Profiles []string `toml:"ok"`
Leaderboard bool `toml:"ok"`
Streamers bool `toml:"ok"`
Countries []string `toml:"ok"`
Log telegraf.Logger `toml:"-"`
}
func (c *Chess) Description() string {
return "a chess plugin"
}
func (c *Chess) SampleConfig() string {
return `
## Indicate if everything is fine
ok = true
`
}
// Init is for setup, and validating config.
func (c *Chess) Init() error {
return nil
}
func (c *Chess) Gather(acc telegraf.Accumulator) error {
if s.Ok {
acc.AddFields("state", map[string]interface{}{"value": "pretty good"}, nil)
} else {
acc.AddFields("state", map[string]interface{}{"value": "not great"}, nil)
}
return nil
}
func init() {
inputs.Add("chess", func() telegraf.Input { return &Chess{} })
}Gathering data
We can now start working on gathering data from the Chess.com API. Within the code above, we have a method called Gather(). This method will contain all of the logic for gathering and passing information from the API to the accumulator of Telegraf. To showcase how to do this, the following will explain our method of gathering leaderboard information from the API, unmarshalling the json response into intermediate structures, and passing that data to the accumulator.
In order to hold the information that is being passed from the call to the API, we created some intermediate structures to hold the data. The call for leaderboard data from the API returns json in the following format:
{
"daily":[
{
"player_id": "integer",
"@id": "URL",
"url": "URL",
"username": "string",
"score": "integer",
"rank": "integer" /* [1..50] */
},
[...]
],
}And so, we constructed two structures to hold the data returned from the API:
type ResponseLeaderboards struct {
PlayerID int `json:"player_id"`
Username string `json:"username"`
Rank int `json:"rank"`
Score int `json:"score"`
}
type Leaderboards struct {
Daily []ResponseLeaderboards `json:"daily"`
}Using the structures defined above, we are able to unmarshal the json information in the response and pass it to the accumulator. The following code replaces our current implementation of the Gather() method:
func (c *Chess) Gather(acc telegraf.Accumulator) error {
if c.Leaderboard {
// Obtain all public leaderboard information from the
// chess.com api
var leaderboards Leaderboards
// request and unmarshall leaderboard information
// and add it to the accumulator
resp, err := http.Get("https://api.chess.com/pub/leaderboards")
if err != nil {
c.Log.Errorf("failed to GET leaderboards json: %w", err)
return err
}
data, err := io.ReadAll(resp.Body)
defer resp.Body.Close()
if err != nil {
c.Log.Errorf("failed to read leaderboards json response body: %w", err)
return err
}
//unmarshall the data
err = json.Unmarshal(data, &leaderboards)
if err != nil {
c.Log.Errorf("failed to unmarshall leaderboards json: %w", err)
return err
}
for _, stat := range leaderboards.Daily {
var fields = make(map[string]interface{}, len(leaderboards.Daily))
var tags = map[string]string{
"playerId": strconv.Itoa(stat.PlayerID),
}
fields["username"] = stat.Username
fields["rank"] = stat.Rank
fields["score"] = stat.Score
acc.AddFields("leaderboards", fields, tags)
}
}
return nil;
}In the code above, we call the leaderboards endpoint from the chess.com API, and store the results in the resp variable, eventually reading all the json information into the data variable. From there, we can then use json.Unmarshal() to take the json information within data and store it within the LeaderBoards structure leaderboards. Once this is done, we iterate through the leaderboards Daily value (which contains an array of ResponseLeaderBoard structs) to transfer the information to the maps fields and tags. For this example, we chose the player ID as the tag used for each fields map. Finally, we supply the string “leaderboards” along with the two maps fields and tags to the acc.AddFields() method to add the data to the accumulator. And with just that, we are collecting information about the leaderboards from Chess.com.
Documenting your plugin with README and making a config file
A boilerplate readme is available in plugins/inputs/EXAMPLE_README.md. We created a sample config file to demonstrate the inputs our plugin offers. Here is our sample config file:
[[inputs.chess]]
# A list of profiles for monitoring
profiles = ["Hikaru", "Firouzja2003", "Mackus"]
# list the leaderboard
leaderboard = true
# List the current streamers on chess.com
streamers = trueWhen creating a README, start by giving a brief description of your plugin and what it is supposed to accomplish. Next section is the configuration. It should outline the TOML and give a description of each field. The sample configuration should also include default values for fields if not set by the user. A sample output should also be included. We used the output of running the default config file.
Making your plugin an external plugin
Next, we decided to make our chess plugin an external plugin. To do this, we followed the Telegraf external plugin guidelines listed in the official repository. External plugins allow for more flexibility and they are built outside of Telegraf that run through an execd plugin. Since our plugin is written using Golang, we will begin using execd go shim. The first part of making an external plugin is to write the plugin itself as described above. Next, move the project to an external repo ensuring to preserve the path structure; ours is plugins/inputs/chess. Copy main.go in the project under the cmd folder. This acts as the entrypoint. It is recommended to only have one plugin per repo. Add the plugin to main.go imports; we added the following code:
_ "github.com/influxdata/telegraf/plugins/inputs/chess"
Next is to add the config file specific to the plugin. This must be separate from the rest of the config for Telegraf. Build the plugin go build -o chess cmd/main.go and test the binary ./chess --config chess.conf. The final step is to configure Telegraf to call the chess plugin.
[[inputs.execd]]
command = ["src/chess", "-config", "chess.conf"]
signal = "none""src/chess" will need to be the path to the executable and "chess.conf" needs to be the path to the config file. The external plugin is now ready for use! The next step is to consider publishing your external plugin to GitHub and open a pull request back to the Telegraf repo to let the Telegraf team know.
Final thoughts
Ultimately, the process of developing a Telegraf plugin is not too difficult. This ease of use and flexibility that Telegraf offers is what drew many of the team toward this technology in the first place. It’s a very versatile and powerful tool. Readers can use our experience and the process we went through to develop their own Telegraf plugins. One could make one for other services, applications, or even games. The possibilities are virtually infinite, so what are you waiting for? Jump in and get started! To learn more about Telegraf, explore the documentation here.
Special thanks to Sebastian Spaink for stepping in and guiding us during this project. Be sure to follow the InfluxData Blog for other projects, tutorials or stories.
