In certain situations, projects might be running multiple Clowder instances, deployed in different labs next to instruments, or across institutions but with similar research. It would be helpful to have metadata about specific datasets in each of these instances appear in searches in other instances. Once a user identifies a datasets they are interested in, we could redirect them to the other instance and still require them to be authenticated and authorized to access the data on that instance. In the case of public datasets, this wouldn't be necessary.
An initial implementation could be to have the Elasticsearch instances registered in a cluster (https://www.elastic.co/guide/en/elasticsearch/reference/current/high-availability.html). While this is meant to be used for high availability it might work across the internet. The admin would have to setup the relationship. Clowder would need to distinguish in the index what resource is local (and as such can be accessed and interacted with) and which one is remote (redirect to other site).
In certain situations, projects might be running multiple Clowder instances, deployed in different labs next to instruments, or across institutions but with similar research. It would be helpful to have metadata about specific datasets in each of these instances appear in searches in other instances. Once a user identifies a datasets they are interested in, we could redirect them to the other instance and still require them to be authenticated and authorized to access the data on that instance. In the case of public datasets, this wouldn't be necessary.
An initial implementation could be to have the Elasticsearch instances registered in a cluster (https://www.elastic.co/guide/en/elasticsearch/reference/current/high-availability.html). While this is meant to be used for high availability it might work across the internet. The admin would have to setup the relationship. Clowder would need to distinguish in the index what resource is local (and as such can be accessed and interacted with) and which one is remote (redirect to other site).