Wolfram Function Repository: Open Access Platform for Wolfram Language Extensions

Hey Habr! I present to your attention the translation of the post by Stephen Wolfram "The Wolfram Function Repository: Launching an Open Platform for Extending the Wolfram Language".

Wolfram Function Repository: Open Access Platform for Wolfram Language Extensions

Prerequisites for the solvency of the Wolfram language

Today we are on the verge of great achievements together with the programming language Wolfram Language. Just three weeks ago we launched free Wolfram engine for developersto help our users integrate the Wolfram Language into their large software projects. Today we are launching wolfram function repository, in order to provide a coordinated platform for functions created to extend the Wolfram language, and we also open a function repository for anyone who can contribute to the development of our software product.

The Wolfram Function Repository is something made possible by the unique characteristics of the Wolfram Language not only as a programming language, but also as full scale computational language. In traditional programming languages, adding significant new features usually involves creating entire additional libraries that may or may not work when used together. However, in the Wolfram Language so much is already built into the language itselfthat it is possible to significantly expand its functionality, simply by adding new features that immediately integrate into the coherent structure of the entire language.

For example, the Wolfram Functions Repository already contains 532 new features structured in 26 thematic categories:

Wolfram Function Repository: Open Access Platform for Wolfram Language Extensions

Likewise, more than 6000 standard functions, built into the Wolfram language, each function from the repository has a documentation page with a detailed description of them and examples of how they work:

Wolfram Function Repository: Open Access Platform for Wolfram Language Extensions

To go to the page, copy the above object (function BLOB), paste it into the input line and then run the function - it is already built into the Wolfram language and is supported by default, starting with 12.0 versions:

Wolfram Function Repository: Open Access Platform for Wolfram Language Extensions

It should be noted here that when processing LogoQRCode You do not need, for example, to set up an "image processing library" - since in the Wolfram Language we have already implemented a consistent and carefully algorithmized way image processing, which can be immediately processed by various graphic functions of the language:

Wolfram Function Repository: Open Access Platform for Wolfram Language Extensions

I hope with the support wonderful and talented community, which has been growing and expanding (based on the Wolfram Language) over the past few decades. The Wolfram Function Repository will allow for the foreseeable future to greatly expand the range of (perhaps potentially significant, specialized in various areas of science and technology) functions available in the language. Thus, it becomes possible to use both the content of the language (its built-in functions) and development principles, which are implemented on the basis of the language. (It should be noted here that the Wolfram Language already has more than 30-year history of development and stable growth).
Inside the functions from the repository, there can be small or rather voluminous code fragments written in the Wolfram Language. For example, these could be calls external APIs and services or external libraries in other languages. The unique thing about this approach is that when you move on to user-level functionality, there will be no possible inconsistencies due to the fact that this approach is built on the agreed structure of the Wolfram Language - and each function will automatically work correctly - exactly as she should.
The shell and programming structure of the Wolfram Functions Repository is set up in such a way that everyone can contribute to the common cause in the most simple and convenient way for him - in fact, just by filling in a notepad text file (with nb extension) WL. Built-in automatic functions allow you to check for new functions added to the repository in order to ensure their integration into the language. Our company is betting on a wide range of users who can integrate their functions into the language, and not on the great complexity of new functions - and although there is a verification process, we do not insist on something like painstaking design analysis or strict standards for the completeness and robustness of new user features, as opposed to the more rigorous feature checks we use built into the core of the language.

There are many trade-offs and details in this approach, but our goal is to optimize the Wolfram feature repository both for the convenience of users and so that new user features make a meaningful contribution to the evolution of the language. As we grow, I have no doubt that we will have to invent new methods of processing and testing the functions that are built into the repository, and this is not the least to organize a large number of functions and find those that users need. However, it cannot but be encouraging that the path we have chosen is a good start. I'm on my own added some features to the original base. Many of them are based on code that I have personally developed over quite a long time. And I spent only a few minutes to send them to the repository. Now that they're in the repository, I can finally use these features immediately and whenever I need them, without having to worry about searching for files, downloading packages, and so on.

Increase efficiency while reducing costs

Even before the Internet, there were ways to share Wolfram Language code (our first major centralized project was MathSource, created for Mathematica in 1991 based on CD-ROM, etc.). Of course, the approach proposed for implementation based on the Wolfram function repository is a more powerful and reliable tool for implementing the above tasks.

For more than 30 years, our company has worked hard to maintain the integrity of the structure of the Wolfram language, and this is very important for the Wolfram language to become not just a programming language, but also complete computing language. And thus, the essence of the approach to implementing the Wolfram Feature Repository is to use a unified approach to programming and developing new features that are sequentially added and fit into the framework of the language for the possibility of its development and co-evolution.

Various computational processes take place in the implementation structure of each function. It should be noted here that in this case it is necessary that the function has a clear and uniform appearance and visual readability for the user. In this context, the built-in functions of the Wolfram Language are represented by more than 6000 consecutive examples of how functions should be properly programmed (these are our live programming video reviews, which include hundreds of hours of the process of creating sample programs). This approach ultimately makes the Wolfram Function Repository capable of working well, precisely the structural nature of the Wolfram Language with its large number of additional and varied libraries that are already built into the language. For example, if you have a function that processes images, or sparse arrays, or molecular structuresand geographical data or some others - the language already has their agreed symbolic representation, and thanks to this, your function immediately becomes compatible with other functions in the language.

Creating a repository that works really well is an interesting meta-programming challenge. For example, an excess of restrictions in the program will not be able to achieve the required unification and universality of the algorithm. Just as with an insufficient number of functional restrictions, you will not be able to implement a sufficiently correct sequence of algorithm execution. Several previous examples of the implementation of a compromise of these approaches implemented by our company worked quite stably - these are: Project Wolfram Demos, launched in 2007 and now online online with over 12000 user interactive demos. IN wolfram database there are more than 600 ready-made databases that can be used in the Wolfram Language, and Wolfram neural network repository is replenished with new neural networks almost every week (now there are already 118) and they are immediately connected through the function netmodel in the Wolfram Language.

All of the above examples have a fundamental feature - the objects and functions assembled in the project have a very high degree of structuring and distribution of processes. Of course, the detail of the structure of what is a demo or a neural network or something else can vary greatly, but the fundamental structure for any current repository always remains the same. So what is your opinion, dear user, about creating such a repository that adds extensions to the Wolfram language? The Wolfram Language is designed to be extremely flexible, so it can be extended and modified in any way. This circumstance is extremely important for the ability to quickly create various large-scale software projects in the Wolfram Language. It should be noted here that with the growth of the flexibility of the language, the cost of projects implemented in such a language will inevitably increase. This is because the more the user uses such a language, the more dedicated functionality they get, but we should not forget that this approach can also have negative sides in terms of the inability to ensure consistent consistency of program modules.

In traditional programming languages, there is a common problem with libraries - if you use one library, for example, then the code works correctly, but if you try to use several libraries, there is no guarantee that they will interact correctly with each other. Also, in traditional programming languages ​​- unlike a full-fledged computing language - there is no way to guarantee control over the presence of consistent built-in representations for any functions or data types other than their basic structures. But, in fact, the problem is even bigger than it seems at first glance: if someone is building a large-scale vertical of functionality, then without the huge cost of centralized programming of the project that we have invested in the Wolfram language, it is impossible to achieve consistency. Therefore, it is important that all program modules always work together correctly.

Thus, the idea behind the Wolfram Function Repository is to avoid the problem mentioned above—simply adding extensions to the language in the form of relatively small pieces of code through individual functions that are easier to develop as consistent modules. However, there are programming features that cannot be made convenient with individual functions (and our company is going to release an optimized software algorithm in the near future to help implement large-scale software packages). However, based on the functions already built into the Wolfram Language, there are many possibilities for programming that are implemented on the basis of individual functions. The idea here is that with relatively little programming effort, a number of new and very useful features can be created that will provide sufficient design coherence, while being well aligned with each other, and also, in addition, they will be able to easily and widely used in the language in the future.

This approach is definitely a compromise. Given the implementation of a larger package, one can imagine a whole new world of functionality that will become extremely powerful and useful. If there is a need to get new functionality that will fit into everything else, but at the same time you are not ready to spend huge efforts on the development of the project, this, unfortunately, can lead to a decrease in the scale of your project. The idea behind the Wolfram Functions Repository is to provide functionality to a defining part of a project, this approach allows you to add powerful functionality while also making it easier to maintain a good programming project consistency.

Help with adding custom functions to the functions repository

Our team has worked hard to make contributing to Wolfram Repository features easy enough for users. On the desktop (already in 12.0 versions), you can simply go through the tabs of the main menu in sequence: File > New > RepositoryItem > Function Repository Item and you will get "Definition Notebook” (programmatically inside the workbench. You can also use the analog function - CreateNotebook["FunctionResource"]):

Wolfram Function Repository: Open Access Platform for Wolfram Language Extensions

There are two main things you will need to do: first, actually write the code for your function, and second, write documentation that illustrates how your function should work.
Click the "Open Sample" button at the top to see an example of what you need to do:

Wolfram Function Repository: Open Access Platform for Wolfram Language Extensions

In fact, you are trying to create something that looks like a built-in function in the Wolfram Language. Except that it can do something much more specific than a built-in function. At the same time, expectations regarding its completeness and reliability will be much lower.
You need to give your function a name that follows the Wolfram Language's function naming conventions. In addition, you will need to develop documentation for your function, similar to the built-in functions of the language. I will talk about this in more detail later. For now, just notice that in the row of buttons at the top of the definition notebook file, there is a button "Style Recommendations", which explains what to do, and a "Tools" button, which provides a toolkit for formatting your function's documentation.
When you are sure that everything is filled in properly and you are ready, click the "Check" button. It is perfectly normal that you have not figured out all the details yet. So the "Check" function will automatically run and do a lot of style and consistency checks. Often, it will immediately prompt you to confirm and accept the corrections (For example: "This line must end with a colon," and it will prompt you to enter a colon). Sometimes she will ask you to add or change something yourself. We will be constantly adding new features to the automatic functionality of the "Check" button, but its main purpose is to ensure that everything you submit to the features repository already exactly matches as many style guidelines as possible.

Wolfram Function Repository: Open Access Platform for Wolfram Language Extensions

So, after running "Check" you can use "Preview" ("Preview"). "Preview" creates a preview of the documentation page you have defined for your function. You can also create a preview for a file created on your computer or for a file located in the cloud storage. If, for some reason, you are not satisfied with what you see in the preview, just go back and make the necessary corrections, and then click the Preview button again.
You are now ready to commit your function to the repository. The Deploy button gives you four options:

Wolfram Function Repository: Open Access Platform for Wolfram Language Extensions

The important thing at this step is that you can submit your function to the Wolfram Functions Repository so that it is available to any user. In this case, you can also place your function for a limited number of users. For example, you can create a feature that is hosted locally on your computer so that it is available when you use that particular computer. Or you can post it in your cloud accountso that it is available to you when you are connected to the cloud. You can also publicly host (deploy) the feature through your cloud account. That way, it won't be in Wolfram's central feature repository, but you can give someone a URL that will allow them to get your feature from your account. (In the future, we will also maintain central repositories throughout our company).

So, let's say you want to actually submit your function to the Wolfram Function Knowledge Base. To do this, you click the "Submit" button to the repository. So what is happening right now? Your application is immediately queued for review and approval by our dedicated team of curators.

As your application progresses through the approval process (which usually takes a few days), you will receive updates on the status of your application, and possibly suggestions on how to further use it. But once your feature is approved, it will immediately be published to the Wolfram Feature Repository and made available for anyone to use. (And it will show up in news digests of new features etc.)

What should be in storage?

It should be noted that our company has very high standards for completeness, reliability and overall quality, of the 6000+ features that we have already built into the Wolfram language over the past 30+ years, all meet the above requirements. The goal of the Wolfram Functions Repository is to take advantage of all the structure and functionality that already exists in the Wolfram Language in order to add as many much lighter (that is, higher performance) functions as possible.

Of course, the functions in the Wolfram Functions Repository must conform to the design principles of the Wolfram Language—so that they can fully interact with other functions and the user's expectations of how the functions should work correctly. However, the functions do not have to have the same completeness or reliability.

In built-in functions of the Wolfram language, we work hard to make program functions as generic as possible. That said, when you're in the Wolfram Functions Repository, there's nothing wrong with having a function that just handles some very specific but useful case. For example, the function SendMailFromNotebook can receive files in one specific format and create mail in one specific way. PolygonalDiagram creates charts with only certain colors and labels, etc.

Another point about inline functions is that our company makes every effort to handle all non-standard cases, to properly handle invalid input, and so on. In a function repository, it is perfectly normal to have a special function that handles the main cases of solving a problem and ignores all the rest.

Obviously it's better to have functions that do more and do it better, but optimizing for a function repository - as opposed to the built-in functions of the Wolfram language - should have more functions tied to more functions, rather than delving into the implementation processes of each specific function.

Now let's look at an example of testing features in a repository. Consistency expectations for such functions are naturally much lower than for built-in language functions. This is all the more true in cases where functions depend on external resources, such as APIs, it is important to constantly conduct sequential tests, which automatically happens inside the verification algorithms. In the nb file, you can explicitly define definitions (in the "More Information" section) and specify as many tests, defined either by input and output lines or full character objects like VerificationTestas much as you need. In addition, the system is constantly trying to turn the documentation examples you provided into a verification process (and sometimes this can be quite resource-intensive, for example, for a function whose result depends on random numbers or time of day).

As a result, there will be a number of implementation complexities in the functions repository. Some will be just one line of code, others may be thousands or tens of thousands of lines long, probably using many helper functions. When is it worth adding a function that requires very little code to define? Basically, if a function has good mnemonic name, which users would readily understand if they saw it in a piece of code, then it can already be added. Otherwise, it's probably better to just re-append code to your program each time you need to use it.

The main purpose of the feature repository (as its name suggests) is to introduce new features into the functionality of the language. If you want to add new data or new entities, use Wolfram Data repository. But what if you want to introduce new kinds of objects for your calculations?

Actually there are two ways. You may want to introduce a new object type to be used in new functions in the functions repository. And in this case, you can always just write down its symbolic representation and use it when inputting or outputting functions in the functions repository.

But what if you want to represent an object and then determine, through existing functions in the Wolfram Language, which should work with it? To do this, the Wolfram Language has always had a lightweight mechanism called UpValues. With some restrictions (especially for functions that can't evaluate their arguments), the function repository allows you to simply submit a function and define values ​​for it. (In order to increase the expectation of consistency when creating a new important construct that is fully integrated throughout the Wolfram Language, this is usually a very important procedure that cannot be achieved only by increasing the cost of the project and this is what our company does as part of projects for the long-term development of the language, this task is not a goal that is set as part of the development of the repository).

So, what can be in the function code in the function repository? Everything built into the Wolfram Language, of course (at least if it doesn't represent угрозы for security and the performance of the program itself as a computing environment) as well as any function from the function repository. However, there are other possibilities of functioning: a function in the function repository can call an API, or in Wolfram Cloudor from another source. Of course, there are some risks associated with this. Because there are no guarantees that the API will not change, and the function in the function store will stop working. To help identify these issues, there is a note on the documentation page (under "Requirements") for any feature that relies on more than just the built-in functionality of the Wolfram Language. (Of course, when it comes to real data, there can be problems even with such functionality - because real world data is constantly changing, and sometimes even changing their definitions and structure.)

Should all code for the Wolfram Functions Repository be written in the Wolfram language? Definitely, the code inside the external API should not be written in the Wolfram language, which actually does not even make the language code. In fact, if you find a function in almost any external language or library, you can create a wrapper that allows you to use it in the Wolfram Function Repository. (Usually you should use the built-in functions for this. ExternalEvaluate or ExternalFunction in Wolfram language code.)

So what's the point of doing this? In essence, this allows you to use the entire Wolfram Language integrated system and its entire unified set of programming features. If you get the base implementation from an external library or language, then you can use the rich symbolic structure of the Wolfram Language to create a handy top-level function that allows users to easily use any functionality already implemented. At the very least, this should be possible in an ideal world where all the building blocks of library loading and the like exist, in which case they would be resolved automatically by the Wolfram Language. (It should be noted that in practice there may be problems with setting up external languages on a specific computer system, and in cloud storage, additional security issues may arise).

By the way, when you first analyze generic external libraries, they often seem too complex to be covered by just a few functions, but in many cases, most of the complexity comes from creating the infrastructure required for the library and all the functions to support it. . However, when using the Wolfram Language, the framework is usually already built into packages, and so it is not necessary to detail all of these support functions, but only to create functions for a few "top-most" application-oriented functions in the library.

"Ecosystem" of the knowledge base

If you've written functions that you use all the time, submit them to the Wolfram Functions Repository! If this does not end up with something more (language development), then it will be much more convenient for you to use functions for personal use. However, it is logical to assume that if you use the features regularly, other users may also find them useful.

Naturally, you may find yourself in a situation where you are not able - or do not want - to share your functions or in case of access to private information resources. Even in such cases, you can simply deploy the features in your own cloud account, specifying the rights access to them. (If your organization has Wolfram Enterprise private cloud, it will soon be able to host its own private feature repository that you can administer from within your organization and force view views or not for third-party users.)

The functions you submit to the Wolfram Functions Repository don't have to be perfect; they just have to be useful. It's a bit like the "Bugs" section of the classic Unix documentation - the "Definitions Section" has an "Author's Notes" section where you can describe limitations, issues, and the like that you already know about your function. Also, when you submit your feature to the repository, you can add submission notes, which will be read by a dedicated team of curators.

Once a feature is published, its page always has two links at the bottom: "Submit a message about this feature" and "Discuss in the Wolfram Community". If you attach a flag (for example, report bugs to me), you can check the box that says that you want your message and contact information to be passed to the author of the function.

Sometimes it becomes necessary to simply use functions from the Wolfram Function Repository, such as built-in functions, without looking into their code. However, if you want to "look inside", there is always a Notepad button at the top. Click on it and you'll get your own copy of the original definition notebook that was submitted to the functions repository. Sometimes you can just use it as an example for your needs. However, you can also develop your own modification of this function. You may want to put these functions you found from the repository on your computer or in your account in your aphid cloud storage, you may want to submit them to the functions knowledge base, perhaps as an improved, augmented version of the original function.

In the future, we plan to support Git-style forking for feature repositories, but for now we're trying to keep it simple and we always have only one generally accepted version of each feature built into the language. More often than not (if the developers don't stop maintaining the features they've developed and respond to user messages), the original author of the feature takes control of updates to it and submits new versions, which are then reviewed and, if they pass the review process, published in the language.

Consider the question of how the "versioning" of the developed functions works. Right now, when you use a function from the functions repository, its definition will be permanently stored on your computer (or in your cloud account if you use the cloud). If a new version of a function is available, the next time you use it, you will receive a message notifying you of this. And if you want to update a feature to a newer version, you can do so with the command resource update. (The "Function Blob" actually stores more versioning information, and we plan to make this more accessible to our users in the future.)

One of the great things about the Wolfram Functions Repository is that any Wolfram Language program anywhere can use functions from it. If the program appears in notepad, it is often convenient to format the functions of the repository as functions of easily readable "function binaries" (perhaps with an appropriate set of versions).

You can always refer to any function of the function repository using a text ResourceFunction[...]. And this is very convenient if you write code or scripts directly for the Wolfram Engine, for example, with using an IDE or a text code editor (Special note that the functions repository is fully compatible with Free Wolfram Engine for Developers).

How does it work?

Inside the functions in the Wolfram repository, this is possible using exactly the same resource systems bases, as in all our other existing repositories (data store, Neural Net Repository, collection of demo projects etc.), like all other Wolfram system resources, resource function ultimately based on function ResourceObject.

Consider resource function:

Wolfram Function Repository: Open Access Platform for Wolfram Language Extensions

Inside you can see some information using the function Information:

Wolfram Function Repository: Open Access Platform for Wolfram Language Extensions

How does setting the resource function work? The simplest is the purely local case. Here's an example that takes a function (in this case, just a pure function) and defines it as the resource function for a given session in the program:

Wolfram Function Repository: Open Access Platform for Wolfram Language Extensions

Once you've made the definition, you can use the resource function:

Wolfram Function Repository: Open Access Platform for Wolfram Language Extensions

Note that this function blob has a black icon Wolfram Function Repository: Open Access Platform for Wolfram Language Extensions. This means that the BLOB refers to the in-memory resource function defined for the current session. A resource feature that is permanently stored on your computer or cloud account has a gray icon Wolfram Function Repository: Open Access Platform for Wolfram Language Extensions. And for the official resource function, there is an orange icon in the Wolfram Function Repository Wolfram Function Repository: Open Access Platform for Wolfram Language Extensions.

So what happens when you use the Deploy menu in the definition notebook? First, it takes all the definitions in the notepad and creates a character from them. ResourceObject). (And if you are using a text-based IDE or program, then you can also explicitly create ResourceObject)

Local deployment of a function from the repository on your computer is performed using the command LocalCache for resource object to save it as LocalObject on your file system. Deployment to a cloud account is performed using the command CloudDeploy for a resource object, while public deployment in the cloud is CloudPublish. In all cases ResourceRegister is also used to register the resource function name, so that ResourceFunction["name"] would work.

If you click the Submit button for the Function Repository, then what happens under it ResourceSubmit called on the resource object. (And if you are using a text input interface, you can also call ResourceSubmit directly.)

By default, the submission is made under the name associated with your Wolfram ID. But if you are submitting an application on behalf of a development team or organization, you can set a separate publisher ID and instead use it as the name to interact with your views.

After you have submitted any of your functions to the functions knowledge base, it will be queued for verification. If you receive comments in response, they will usually be in the form of a text file with additional "comment cells" added. You can always check the status of your application by going to resource system contributor portal. But once your feature is approved, you will be notified (via email) and your feature will be placed in the Wolfram feature repository.

Some subtleties at work

At first glance, it may seem that you can just take the definition pad and put it verbatim in the functions repository, however, in fact, there are quite a few subtleties - and processing them requires quite complex meta-programming, processing symbolically as the code that defines the function , and Notepad definition itself. Much of this happens internally, i.e. "behind the scenes", but it can have some implications that are worth understanding if you're going to be contributing to the feature knowledge base.

First immediate subtlety: When you fill out the "Definitions Notebook", you can simply refer to your function everywhere using a name like MyFunction, which looks like a normal function name in the Wolfram Language, but for function repository documentation this is replaced by ResourceFunction["MyFunction"] is exactly what users will actually use when working with the function.

The second subtlety is that when you create a resource function from the Definition Notepad, then all the dependencies involved in the function definition must be fixed and explicitly included. However, to ensure that the definitions remain modular, you need to put everything in a unique namespace. (It goes without saying that functions that do it all, are in the functions repository.)

You will usually never see any trace of the code used to set up this namespace. But if, for some reason, you call an unexecuted symbol inside your function, you will see that symbol is in the internal context of the function. However, when processing the Definition Notepad, at least the character corresponding to the function itself adjusted for best display as a functional BLOB rather than a raw character in the internal context.

The feature repository is for defining new features. And these functions can have options. Often these options (for example, Method or ImageSize) will be usable for built-in functions as well as those for which built-in symbols already exist. But sometimes a new feature may require new options. In order to maintain modularity, it is necessary that these parameters be symbols defined in a unique internal context (or something like entire resource functions, that is, by themselves). For simplicity, the functions repository allows you to specify new options in definitions as strings. And for the convenience of the user, these definitions (assuming they used OptionValue и OptionsPattern) are also processed so that when using functions, parameters can be specified not only as strings, but also as global symbols with the same names.

Most functions just do what they're supposed to do every time they're called, but some functions need to be initialized before they can run in a particular session - and there's an Initialization section in the Definition section to solve this problem.

Functions from the repository can use other functions that are already in the repository, in order to set up definitions for a function repository that include two (or more) functions that reference each other, you need to deploy them in your program session so that you can reference on them like ResourceFunction["name"], then you can create the combinations of these functions you need examples (I didn’t understand) and add a new function to the repository based on those already posted earlier. (or already or earlier - both words are clumsy)

Development prospects. What should happen when the repository gets really big?

Today we are just launching the Wolfram Functions Repository, but as time goes on we expect it to (remove) its scope and functionality can increase dramatically, and as it grows in development, there will be various problems that we can already foresee can arise.

The first problem concerns function names and their uniqueness. The function repository is designed in such a way that, as with built-in functions in the Wolfram Language, any given function can be referenced simply by specifying its name. But this inevitably means that function names must be globally unique throughout the repository, so that, for example, there can only be one in a repository ResourceFunction["MyFavoriteFunction"].

At first glance, this may seem like a big problem, but it's worth understanding that this is basically the same problem as for things like internet domains or social media markers. And the fact is that it is simply necessary to have a registrar in the system - and this is one of the roles that our company will perform for the knowledge base of Wolfram functions. (For private versions of the repository, administrators can be their registrars.) Of course, an Internet domain can be registered without having anything on it, but in a functions repository, a function name can only be registered if there is an actual function definition.

Part of our role in managing the knowledge base of Wolfram functions is to ensure that the name chosen for the function is logical given the function's definition, and that it conforms to Wolfram Language naming conventions. We have over 30 years of experience naming built-in functions in the Wolfram Language, and our curation team (curators?) will bring this experience to the function repository as well. Of course, there are always exceptions. For example, it might seem preferable to have a short name for some function, but it's better to "defend" with a longer, more specific name, because it's less likely to run into someone wanting to make a similar function name in the future.

(It should be noted here that simply adding some member tag to disambiguate functions will not have the desired effect. Because, if you do not insist on always assigning a tag, you will need to define a default tag for any given function, as well as allocate author tags, which will again require global coordination.)

As the knowledge base of Wolfram functions grows, one of the problems that will most likely arise is the discoverability of functions, for this the system provides search function (and definition files may include keywords, etc.). For built-in functions in the Wolfram Language, there are all kinds of cross-references in the documentation that help "advertise" the functions. Functions in the function repository can reference built-in functions. But what about the other way around? To do this, we're going to experiment with different schemes to introduce repository functions in the documentation pages for built-in functions.

For built-in functions in the Wolfram Language, there is a so-called detection level provided by a network of "help pages", which provide organized lists of features related to specific areas. It is always difficult to properly balance manual pages, and as the Wolfram language grows, manual pages often need to be completely reorganized. It's easy to place functions from a repository into broad categories and even break those categories down consistently, but it's much more valuable to have well-organized language reference pages. It is not yet clear how best to create them for the entire feature knowledge base. For example, CreateResourceObjectGallery in the functions repository, anyone can host a web page containing their "selections" from the repository:

Wolfram Function Repository: Open Access Platform for Wolfram Language Extensions

The Wolfram Function Repository is set up as a persistent function repository, where any function in it will always work. Of course, new versions of functions may also appear, and we expect that some functions will of course become obsolete over time. The functions will work if they are used in programs, but their documentation pages will link to newer, more advanced functions.

The Wolfram Functions Repository is designed to quickly get new features and explore new ways to use the Wolfram language. With a lot of optimism, we hope that some of what has been explored in the feature repository will eventually make sense as built-in parts of the core Wolfram Language. Over the past decade, we've had a similar set features that were originally introduced in Wolfram | Alpha. And one of the lessons learned from this experience is that it takes a lot of work to achieve the standards of quality and consistency that we emphasize in everything that is built into the Wolfram language, which is often more difficult than the initial efforts to implementation of the idea. Even so, a function in the function knowledge base can serve as a very useful proof of concept for a future function that could eventually be built into the Wolfram language.

The most important thing here is that a function in the functions repository is something that is available to every user for use right now. Perhaps a built-in language feature could be much better and more performant, but a feature repository would allow users to immediately access all new features. And, most importantly, this concept allows everyone to add any of their new features.

Earlier in the history of the Wolfram language, this idea would not have worked as well as it does now, but at this stage so much effort has already been put into the language, as well as such a deep understanding of the principles of language design, that it now seems very possible that a large community of users will add features that will maintain design consistency to make them useful to a wide range of users.

There is an incredible spirit of talent(?) in the Wolfram Language user community. (Of course, this community includes many leading R&D experts in a wide variety of fields.) I hope that the Wolfram Feature Repository provides an effective platform for uncovering and spreading this spirit of talent. Only together can we create something that greatly expands the scope to which the Wolfram computational paradigm can be applied.

We've come a long way in the Wolfram language in 30+ years. Now together, let's go even further. I urge all respected users of the Wolfram language around the world to use the functional repository as a platform for this, as well as such a new software project as the Free Wolfram Engine for developers.

Source: habr.com

Add a comment