Building a Custom YAML Loader for Webpack

One of our internal tools at RedBit uses yaml to provide structured data. We chose yaml because it's more human-friendly than json, and the data in question is created and modified by humans during development. Applications need to consume the data as JavaScript objects, so we need to convert it from yaml to json. We could do the conversion at runtime but that would affect performance, possibly to the point of degrading the user experience. Instead, we chose to convert the data during the build process, which is controlled by webpack. This required a custom webpack loader. I won't describe the implementation of our internal tool in detail because it wouldn't be particularly helpful or relevant. Instead, I'll show you how to build a simple yaml loader that you can modify to suit your own needs.

The Loader

// yaml-loader.js

const { getOptions } = require('loader-utils');
	const validate = require('schema-utils');
	const yaml = require('js-yaml');

	const loaderOptionsSchema = {
	  type: 'object',
	  properties: {
	    commonjs: {
	      description: 'Use CommonJS exports (default: false)',
	      type: 'boolean',
	    },
	  },
	};

	module.exports = (loaderContext, source) => {
	  const callback = loaderContext.async();
	  const loaderOptions = getOptions(loaderContext) || {};

	  validate(loaderOptionsSchema, loaderOptions, {
	    name: 'yaml-loader',
	    baseDataPath: 'options',
	  });

	  const { commonjs = false } = loaderOptions;

	  try {
	  	const data = yaml.load(source);

	  	// At this point you may perform additional validations
	  	// and transformations on your data...

	  	const json = JSON.stringify(data, null, 2);
	  	const ecmaFileContents = `export default ${json};`;
	  	const cjsFileContents = `module.exports = ${json};`;
	  	const fileContents = commonjs ? cjsFileContents : ecmaFileContents;
	  	callback(null, fileContents);
	  } catch (error) {
	  	callback(error);
	  }
	};

We begin by defining a schema for the loader options so we can validate them with schema-utils. The schema is an object that describes properties that you can set when you integrate the loader in your webpack config:

	const loaderOptionsSchema = {
	  type: 'object',
	  properties: {
	    commonjs: {
	      description: 'Use CommonJS exports (default: false)',
	      type: 'boolean',
	    },
	  },
	};

In this case, we have one option, 'commonjs', which is a boolean. If the 'commonjs' option is 'true', the loader will generate JavaScript files that use CommonJS modules (e.g. 'module.exports'). Otherwise, the loader will use ECMA modules (e.g. 'export default'). You will likely want ECMA modules in most modern web applications, but the option gives you some flexibility to work in other environments. (Note that the loader itself is written as a CommonJS module because it always runs in Node. Using CommonJS avoids some compatibility issues with ECMA modules in older versions of Node.)

Next, we have the loader function itself:

	module.exports = (loaderContext, source) => {
	  const callback = loaderContext.async();
	  const loaderOptions = getOptions(loaderContext) || {};

	  validate(loaderOptionsSchema, loaderOptions, {
	    name: 'yaml-loader',
	    baseDataPath: 'options',
	  });

	  const { commonjs = false } = loaderOptions;

	  try {
	  	const data = yaml.load(source);

	  	// At this point you may perform additional validations
	  	// and transformations on your data...

	  	const json = JSON.stringify(data, null, 2);
	  	const ecmaFileContents = `export default ${json};`;
	  	const cjsFileContents = `module.exports = ${json};`;
	  	const fileContents = commonjs ? cjsFileContents : ecmaFileContents;
	  	callback(null, fileContents);
	  } catch (error) {
	  	callback(error);
	  }
	};

​This function accepts two arguments. 'loaderContext' is the context in which the loader runs. We'll use this to obtain some information about the loader, including the options. 'source' is the contents of the input file as a string (a yaml string in this instance). The function performs the following tasks:

1. Call 'loaderContext.async()' to tell the loader that the process will run asynchronously. 'loaderContext.async()' returns a callback that we'll use to pass the results of the process back to the loader.
2. Obtain the loader options by calling 'getOptions(loaderContext)', which is a function provided by loader-utils. We default the return value of 'getOptions' to an empty object literal in case the webpack config doesn't include the options hash.
3. Validate the loader options against the schema we created earlier. This will throw an error if the options aren't specified correctly in the webpack config. Unpack the options, if desired.
4. Parse the 'source' string. We're using js-yaml for this.
5. At this point the data is parsed and you can perform additional validations and transformations on it.
6. Json-serialize the data using 'JSON.stringify()'. Set the indentation according to your preferences (2 spaces in this case).
7. Create the file contents for ECMA and CommonJS modules by appending the serialized json string to the export statement.
8. Execute the callback with the appropriate file content string based on the 'commonjs' option.

The result of this process will be that you will be able to import a yaml file in your JavaScript code, and its contents will be made available as a JavaScript module instead of a yaml string.

	import data from './data.yaml';

	for (key in data) {
	  console.log(`The value for key '${key}' is ${data[key]}`);
	}

Integration

Integration with webpack is similar to any other loader. Add a new rule to the 'module.rules' array in your webpack config. The rule should 'test' files for the '.yaml' file extension, 'exclude' any files in the 'node_modules' directory and 'use' your custom yaml loader:

	// webpack.config.js
 module: {
      rules: [
        {
          test: /\.yaml$/,
          exclude: /node_modules/,
          use: {
            loader: path.resolve('./yaml-loader'),
            options: {
              commonjs: false,
            },
          },
        },
      ],
    },

You can use this loader for simple yaml files, or as a starting point for a more complex loader of your own.