Plugins Overview
What a Plugin Is?
Plugin is a component that extends TargetProcess functionality.
Plugin is installed separately from TargetProcess and communicates with TargetProcess via Message Bus. Within plugin you can react to events in TargetProcess in different ways. You have the ability to create/update/delete any item in TargetProcess. Another ability is to write a synchronizable plugin which will execute some actions regularly.
Plugin can have several profiles. Profile is a set of settings for plugin instance. You can create/update/delete profiles on TargetProcess UI. For example, you want to synchronize TargetProcess bugs and Bugzilla bugs. You should create a profile which contains connection settings to Bugzilla. If you want to have different settings, you need to have several profiles.
Each plugin profile has its own storage. In this storage you can store any data you need. For example, in case of Bugzilla plugin, you will probably need to store synchronized bugs ids. You may choose whether to use TargetProcess database for storage or create your own.
Technology Overview
MSMQ should be installed to run and create TargetProcess plugins. MSMQ is a Windows component. MSMQ should be installed automatically when you install TargetProcess. In any case, you can install it manually, just follow the instructions.
NServiceBus is used as a framework for messages communication. You may find documentation and source code of NServiceBus at http://www.nservicebus.com. All features of NServiceBus are available when you write a plugin.
Plugin can be installed and ran in two ways:
- As a Windows Service (Recommended. This way is used for all default TargetProcess plugins).
- As a console application.
Dependency Injection is widely used in a new plugin framework. StructureMap is an IOC container. So all classes available in plugin can be resolved via StructureMap. Recommended way is to use constructor injection.
LinqToSql ORM is used for profile storage manipulations.
Profile
Profile is a main entity of a plugin. When you handle a message, your code is executed in a profile context.
You can check values in context by retrieving IPluginContext instance from StructureMap container.
Almost all objects that you retrieve from StructureMap container depend on context. It means that when you retrieve IStorageRepository object with "Profile1" in context then this class will write to "Profile1" storage. The same is true for profile log. You don't need to choose the profile and then write to its storage. Just use classes provided with the API.
To define a profile in a plugins you should create a class and mark it with Profile attribute.
There can be only one profile class per plugin. You may define any properties you need to edit on UI in this class.
One important note is that profile should have a DataContract.
It means that profile class should be marked with System.Runtime.Serialization.DataContractAttribute and all members you want to edit on UI should be marked with System.Runtime.Serialization.DataMemberAttribute.
[Profile, Serializable]
[DataContract]
public class TwitterProfile
{
...
}
You can validate data before saving profile on UI. Just implement IValidatable interface. If validation fails, you will get response with provided errors.
[Profile, DataContract]
public class TaskCreatorProfile : IValidatable
{
[DataMember]
public int Project { get; set; }
public void Validate(PluginProfileErrorCollection errors)
{
ValidateProject(errors);
}
private void ValidateProject(PluginProfileErrorCollection errors)
{
if(Project <= 0)
{
errors.Add(new PluginProfileError{FieldName = "Project", Message = "Project should not be empty"});
}
}
}
You can create a synchronizable profile. For example, you want to check whether there are some changes in Bugzilla every five minutes.
Implement ISynchronizableProfile interface and receive a specific Tick message on each interval.
[Serializable, Profile, DataContract] public class ProjectEmailProfile : ISynchronizableProfile
public class TickHandler : IHandleMessages<TickMessage>
Profile Storage API
Each plugin may have its own database or you may share the same database among several plugins. You need to provide a connection string in config file in pluginDatabaseConnectionString setting value.
<Tp.Integration.Plugin.Common.Properties.Settings>
...
<setting name="pluginDatabaseConnectionString" serializeAs="String">
<value>Data Source=(local);Initial Catalog=TargetProcess;user id=sa;password=sa</value>
</setting>
...
</Tp.Integration.Plugin.Common.Properties.Settings>
To get access to storage, receive an instance of IStorageRepository class from StructureMap container.
var storage = ObjectFactory.GetInstance<IStorageRepository>();
With this class you can save, update and delete entities in database. This class always works with a current profile, all changes execute on current profile storage.
You just work with storage like with a collection of items.
private readonly IStorageRepository _storageRepository;
public void Handle(ProjectDeletedMessage message)
{
var projectDtos = _storageRepository.Get<ProjectDTO>();
var projectDtoToRemove = projectDtos.FirstOrDefault(x =< x.ID == message.Dto.ID);
_storageRepository.Get<ProjectDTO>().Remove(projectDtoToRemove);
}
When you store some entity in profile storage, it is serialized and then written to database.
DataContractJsonSerializer is used. So you can use all power of Data Contracts to perform custom serialization.
After your plugin comes to production, be careful with changing the signature of stored types. It may break deserialization process.
Profile Log
Each profile has a log associated with it. By default these logs are placed in "Logs" folder. See log4net section in Tp.Integration.Plugin.Common.dll.config for details.
Also general log with all logged information presents in "Logs" folder.
In order to write to profile log you should retrieve IActivityLogger from StructureMap container.
ObjectFactory.GetInstance<IActivityLogger>().Error("Error occured.");
Handle Messages From TargetProcess
New plugins architecture is based on SOA principles. Plugins are hosted separately from TargetProcess and all code executes asynchronously. This brings us to a Message Bus concept.
MSMQ is used as a communication channel. Each plugin has its own message queue. You should specify plugin queue name in the config.
<Tp.Integration.Plugin.Common.Properties.Settings>
<!--.....-->
<setting name="PluginInputQueue" serializeAs="String">
<value>Tp.MyPluginInputQueue</value>
</setting>
<!--.....-->
</Tp.Integration.Plugin.Common.Properties.Settings>
TargetProcess sends messages about changes to this queue. It is created automatically when a plugin starts for the first time.
To communicate with TargetProcess you should specify TargetProcess input queue in the config file:
<Tp.Integration.Plugin.Common.Properties.Settings>
<!--.....-->
<setting name="TargetProcessInputQueue" serializeAs="String">
<value>Tp.InputCommand</value>
</setting>
<!--.....-->
</Tp.Integration.Plugin.Common.Properties.Settings>
Tp.InputCommand is a default value.
When any event occurs in TargetProcess (i.e. UserStory is created), it sends a message to all plugins.
To handle a message you need to implement IHandleMessage<T> interface where T is a message type you want to handle.
Message is sent once for each profile.
public class CreateTasksForNewUserStoryHandler : IHandleMessages<UserStoryCreatedMessage>
{
public void Handle(UserStoryCreatedMessage message)
{
//write your code here.
}
}
Send Commands to TargetProcess
You often want to react on some external changes and perform some actions with TargetProcess in a plugin. There is an ability to send commands to TargetProcess.
ICommandBus is an interface responsible for command sending. You can resolve it from StructureMap container. Here is an example of its usage:
private readonly ICommandBus _bus;
private void CreateTask(string taskName, UserStoryDTO userStoryDto)
{
_commandBus.Send(new CreateTaskCommand(new TaskDTO
{
UserStoryID = userStoryDto.UserStoryID,
Name = taskName,
OwnerID = userStoryDto.OwnerID
}));
}
Sometimes an exception may occur in TargetProcess during command execution (e.g. entity did not pass validation rules).
In this case TargetProcess will send a message TargetProcessExceptionThrownMessage with the exception inside.
Handle this message in you saga, or by message handler, to be notified about exceptions.
public class ExceptionHandler : IHandleMessages<TargetProcessExceptionThrownMessage>
What a Saga Is?
Sometimes you need to handle several operations one by one. In our Bugzilla integration sample, you need to create a bug first and then add comments. In such long-running business transaction a saga should be used.
The sagas (from Icelandic saga, plural sögur) are stories about ancient Scandinavian and Germanic history, about early Viking voyages, the battles that took place during the voyages, about migration to Iceland and of feuds between Icelandic families. They were written in the Old Norse language, mainly in Iceland.
Here is the full information about sagas. The only difference in our plugin system is that
you need to inherit TpSaga<TSagaData> class. There you will find a shortcuts for Send method, SendLocal method, StorageRepository instance and some more.
Important : use this methods for sending commands to TargetProcess.
Important : SagaData class should be unique per each saga.
Let’s consider the following scenario: in TargetProcess we need to create a copy of just created bug with changed name. Sounds simple? Not quite!
The problem is that our plugin will actually receive TWO messages BugCreatedMessage: the first one when original bug created and the second one when a copy was created. So there is a danger of recursion!
Here is a sequence diagram:
And here is the saga:
/// This is an example which demonstrates how to create a multi step scenarious
/// First, we need to derive from TpSaga and specify which message will start whole saga.
/// We will also provide a storage which will keeps all data contained in saga. For now it'll be a CreateBugCopySagaData.
public class CreateBugCopySaga : TpSaga<CreateBugCopySagaData>,
ISagaStartedBy<BugCreatedMessage>
{
/// This one is obligatory. Here we will specify conditions to make sure whether incoming message arrived
/// as a response
public override void ConfigureHowToFindSaga()
{
ConfigureMapping<BugCreatedMessage>(
saga => saga.Id,
message => message.SagaId
);
}
public void Handle(BugCreatedMessage message)
{
if (IsResponse(message)) //check if it's a response.
{
MarkAsComplete(); //we caught
}
else
{
AddCopy(message);
}
}
private void AddCopy(BugCreatedMessage bugCreatedMessage)
{
var dto = bugCreatedMessage.Dto;
dto.Name += " (copy)";
Send(new CreateBugCommand(dto));
}
}
Send local Commands
A good design decision is to separate your code on many independent components. Local messages can help you greatly to achieve this. ILocalBus is a class responsible for local message sending. You can resolve it from StructureMap container. In order to define a local message you should implement IPluginLocalMessage interface.
Let's take Bugizlla plugin as an example. We need to import new bug from Bugzilla to TargetProcess. First of all we create a bug in TargetProcess. Then we need to import comments to this bug. After bug was created in TargetProcess, we send local message with this bug. And define appropriate handler for comments creating.
// Bug creation saga
public class BugImportSaga : TpSaga<BugImportSagaData>,
IAmStartedByMessages<ImportBugToTargetProcessCommand>,
IHandleMessages<BugCreatedMessage>,
IHandleMessages<BugUpdatedMessage>,
IHandleMessages<TargetProcessExceptionThrownMessage>
{
//......
public void Handle(BugCreatedMessage message)
{
// here we retrieve bugzilla bug from saga data.
var bugzillaBug = Data.BugzillaBug;
_logger.InfoFormat("Bug imported. {0}; TargetProcess Bug ID: {1}", bugzillaBug.ToString(), message.Dto.BugID);
SendLocal(new NewBugImportedToTargetProcessMessage { TpBugId = message.Dto.ID, BugzillaBug = bugzillaBug });
MarkAsComplete();
}
//......
}
//Comment import saga
public class CommentImportSaga : TpSaga<BugCommentImportSagaData>, IAmStartedByMessages<NewBugImportedToTargetProcessMessage>
{
// ....
}
public class NewBugImportedToTargetProcessMessage : IPluginLocalMessage
{
public int? TpBugId { get; set; }
public BugzillaBug BugzillaBug { get; set; }
}
Access Web Services
SOAP API usage is deprecated. It will not be improved anymore. Use REST API instead.
You can get access to TargetProcess SOAP web services in your code. They are stored in StructureMap container.
TBD - For full list of available web services see (link).
In order to use web service, provide TargetProcess url, login and password in plugin settings file.<setting name="TargetProcessPath" serializeAs="String"> <value>http://localhost/TargetProcess</value> </setting> <setting name="AdminLogin" serializeAs="String"> <value>admin</value> </setting> <setting name="AdminPassword" serializeAs="String"> <value>admin</value> </setting>
TBD - web service usage example
Profile Initialization
The good thing about plugins is that plugin may continue running even when TargetProcess is down! Plugin will send messages to TargetProcess queue, and TargetProcess will handle them later, after recovering from server failure.
It is really not recommended to make plugin code dependent on TargetProcess directly. For example, if you use direct calls to SOAP or REST in the plugin, you will lost this luxury.
Let's say, we want to create a plugin that imports bug from external system and assign it to some user in TargetProcess. To do that, we need to know UserId. But how can we know UserId without querying TargetProcess? Profile initialization concept is our assistant here.
It is required to preload all needed data from TargetProcess just after profile adding and save the data into local storage. In our example, you get all users from TargetProcess and keep'em up to date by handling UserAdded, UserUpdated and UserDeleted messages.
In order to perform some initialization logic, create a class inherited from NewProfileInitializationSaga or UpdatedProfileInitializationSaga(if you want to reload some data on profile update). After profile adding this saga will be started. And profile will be in "not initialized" state. In "not initialized" state profile do not handle messages from TargetProcess. It stores them in profile storage.
When InitializationSaga is completed (you should call MarkAsComplete() method in your saga), then profile is marked as "initialized" and all stored messages are immediately processed.
Remember that bug in your initialization saga can make your profile never receive messages from TargetProcess, because it will remain in "not initialized" state.
REST is preferrable way to retrieve data from TargetProcess during profile initialization.
public class PluginInitializationSaga : NewProfileInitializationSaga<PluginInitializationSagaData>
{
protected override void OnStartInitialization()
{
try
{
var client = new WebClient {Credentials = new NetworkCredential("user", "user")};
var res = client.DownloadString("http://mycompany.tpondemand.com/api/v1/Users");
var doc = XDocument.Load(new StringReader(res));
var userElements = doc.Elements().Elements();
foreach (var userElement in userElements)
{
var user = new TpUserLite
{
UserId = int.Parse(userElement.Attribute("Id").Value),
UserEmail = userElement.Element("Email").Value
};
StorageRepository().Get<TpUserLite>().Add(user);
}
}
finally
{
MarkAsComplete();
}
}
}
[DataContract]
public class TpUserLite
{
public int? UserId { get; set; }
public string UserEmail { get; set; }
}
public class PluginInitializationSagaData : ISagaEntity
{
public Guid Id { get; set; }
public string Originator { get; set; }
public string OriginalMessageId { get; set; }
}
Plugin Mashups
UI in plugins can be done via Mashups only.
Typical Mashups usage is for profile editor page.
Create Mashups folder in plugin and ProfileEditor folder inside. Put all required mashups (javascript files) into this folder.
Mashups folder should be in the directory where plugin is running. Set "Copy to Output Directory" to "Copy Always" for your mashup files in Visual Studio.
Plugin Custom Commands
Finally, you have written Bugzilla Integration plugin. Now you want to show a link to Bugzilla bug near each bug in TargetProcess. Mashups feature allows you to add custom UI, but where can we get the link to Bugzilla bug? Our Bugzilla plugin certainly knows about that link, but won't say.
So it is required to ask plugin in our mashup code. You can use plugin custom commands to handle this case.
To define a command in the plugin just implement IPluginCommand interface. Each command should have a unique name.
public class GiveMeTheLink : IPluginCommand
To call the command from mashup, use REST API. Create POST query to the following resource:
/api/v1/Plugins/{PluginName}/Commands/{CommandName}.
and posted data will be passed to Execute() method of your command.
Here is an example with jQuery:
$.ajax({
url: /api/v1/Plugins/'Bugzilla Integration'/Commands/GiveMeTheLink
data: null,
success: function() {
alert('Success');
},
error: function() {
alert('Error');
},
type: 'POST',
dataType: "json"
});
Plugin Embedded Commands
The following resources for plugins are available in REST:
api/v1/Plugins |
Return all available plugins |
api/v1/Plugins/{pluginName} |
Get plugin by name |
api/v1/Plugins/{pluginName}/Commands |
Get all available commands for plugin |
api/v1/Plugins/{pluginName}/Commands/{commandName} |
Execute plugin command |
api/v1/Plugins/{pluginName}/Profiles |
GET: get all profiles for plugin POST: add new profile |
api/v1/Plugins/{pluginName}/Profiles/{profileName} |
GET: get profile POST: add or update profile |
How to Install a Plugin
During plugin development, it is handy to start the plugin as a simple console application.
Just start NServiceBus.Host.exe from plugin output directory.
During plugin development you will probably add some new messages and change its format. Since messages are stored in MSMQ and sagas are stored in database, you may get some serialization errors after this changes. In this case clean up plugin queues and database storage and start plugin again.
When plugin is completed, you want to permanently add it to TargetProcess (install it as a windows service). Run the following command:
NServiceBus.Host.exe /install /serviceName:{YourServiceName} /displayName:"{YourDisplayName}" /description:"{YourDescription}"
To uninstall windows service, run the following command:
NServiceBus.Host.exe /uninstall /serviceName:{YourServiceName}
For more details read http://nservicebus.com/GenericHost.aspx.
How to create a database for plugin.
When you start plugin development, you need to create a database for your new plugin. Actually you can use TargetProcess database (just provide its connection string in plugin config), but then you would mix your plugin data with production data.
In order to create database for plugin you need InstallHelper.exe utility.
It comes with NuGet package and located by the following path : {PathToYourPluginProject}\packages\TargetProcessPluginSDK.2.22.0.15001\tools\.
Execute the following command
InstallHelper.exe -installpluginsdatabase /DbConnectionString:"{provide your connection string here}"
This would create database with tables. Use this connection string in your plugin then.
Plugin database consists of the following tables : [Plugin], [Account], [Profile], [ProfileStorage].
If you need to clean the database, just delete from [Plugin] table. Tables have cascade deletion, so all tables would be cleaned up.
Check Tutorial to create your first Plugin easily.
