Think Different

Sunday

Gadgets aplciation with PHP and Shindig server

Shindig - an Apache incubator project for OpenSocial and gadgets

What is Shindig?

Shindig is a new project in the Apache Software Foundation incubator and is an open source implementation of the OpenSocial specification and gadgets specification.

The architectural components of Shindig can be broken down as follows:

Gadget Container JavaScript -- core JavaScript foundation for general gadget functionality. This JavaScript manages security, communication, UI layout, and feature extensions, such as the OpenSocial API.
Gadget Server -- used to render the gadget XML into JavaScript and HTML for the container to expose via the container JavaScript.
OpenSocial Container JavaScript -- JavaScript environment that sits on top of the Gadget Container JavaScript and provides OpenSocial specific functionality (profiles, friends, activities, datastore).
OpenSocial Data Server -- an implementation of the server interface to container-specific information, including the OpenSocial REST APIs, with clear extension points so others can connect it to their own backends.

Gadgets are web-based software components based on HTML, CSS, and JavaScript. They allow developers to easily write useful web applications that work anywhere on the web without modification. They are defined using a declarative XML syntax that is processed by a gadget server into a format that allows them to be embedded into various contexts: standalone web pages, web applications, even other gadgets. A context into which a gadget is embedded is called a gadget container. The container is responsible for managing the gadgets' layout and controls, as well as for supporting various functionality on behalf of the gadget. A gadget may be a simple widget, a reusable component, or a full-blown application, possibly utilizing or communicating with other gadgets.

A gadget and its XML are synonymous. The gadget XML contains all information needed to identify and render a web application.

Metadata. Several pieces of metadata are specified by the gadget developer in the gadget XML, such as author information, gadget title, and description. This data gives hints to the gadget container on how to identify it, in addition to providing data to gadget directories, which are databases that help users find gadgets useful for their needs

Gadget Features. A list of all gadget features that are either required for the gadget to operate, or may optionally be utilized if available. Gadget features are the primary extensibility mechanism employed by gadgets. They often direct a gadget server to make new JavaScript APIs available to rendering code, but may also manipulate the gadget's contents, for example to add extended syntax. Examples of gadget features include OpenSocial (provides gadgets with a rich set of social APIs), dynamic-height (enables gadgets to resize themselves to an appropriate height), and tabs (a UI library facilitating tabular navigation).

User Preferences. These are key/value pairs that form the basis of gadget configuration and persistence. They are most often manipulable by users of the gadget, and are persisted on behalf of a user so that the gadget has access to them across multiple rendering requests. The gadget container is typically responsible for providing their persistence for this data and an interface to edit it.

Message Bundles. Message bundles allow developers to internationalize their gadgets simply by adding name/message mappings corresponding to whatever languages the developer chooses to support. These messages may be accessed programmatically through the core JavaScript APIs provided to all gadgets, or may be statically substituted into code using simple syntax.

Content. Provides the actual HTML, CSS, and JavaScript to be rendered by the gadget. Two delivery mechanisms are supported:

Type HTML gadgets are the most prevalent, and are imbued with the most rich feature set. Code is provided directly in the gadget XML content section for rendering and control flow. This code simply assumes that functionality is available that has been requested from whatever gadget features were declared. In the case of optional-declared features, a simple feature-existence API can and should be consulted to ensure the capability is enabled. The code is processed by a gadget server and rendered in an IFRAME.
Type URL gadgets only specify a base URL. A standard set of parameters are added to this URL by the gadget server, which renders the gadget in an IFRAME. The application to which the URL points must include a referenced JavaScript library using a tag as it renders, to enable gadget APIs to be made available. Type URL gadgets can't take advantage of all features, notably features that manipulate HTML and JavaScript code directly. However, this gadget type has proven highly useful for turning existing web sites or applications into gadgets.

Multiple Content sections may be specified in gadget XML. Each is labeled with one or more optional view identifiers, which allow the gadget to behave or appear differently depending on the context in which it's rendered. This context is provided by the gadget container.

This document describes the gadget XML syntax and how it is processed by a compliant gadget server. In addition, it describes the core JavaScript APIs that must be made available to every gadget as it is rendered. While compliance can be attained by supporting only these core APIs, doing so severely limits a gadget server's usefulness. Gadget servers should support as many features as possible. Widely used, highly recommended features are listed at the end of this document.

Compliance: Gadget Server

To be gadgets-compliant, a server must be able to satisfy a Gadget Rendering Request and a JavaScript Request in the manner described. The server should be able to satisfy the Gadget Metadata Request in the manner described.

Gadget Rendering Request

The core gadget API, this translates gadget XML into content that can be rendered in a browser, typically in an IFRAME.

Inputs:

Gadget XML, typically specified as a URL pointing to a file on the web.
User prefs.
Locale of the end user.
(optional) IgnoreCache processing option.
(optional) Module ID, an integer identifier for the gadget in a given container context. If not provided, this defaults top 0.

Outputs:

HTML, CSS, and JavaScript.

Process:

Fetch the gadget XML.
1. This SHOULD consult a cache to minimize load on the gadget XML hosting server. The specific properties of caching are left to the implementation.
2. The server may honor HTTP caching headers to provide developers with a way to balance the load their servers receive against the frequency with which they can reliably deploy updated content.
3. The server SHOULD honor an IgnoreCache processing option. This causes all built-in spec caches to be ignored, fetching the spec directly from its canonical resource. This feature is very useful while developing a gadget.
  1. HTTP-based request servers SHOULD support this via a URL parameter named nocache.
  2. The server MAY implement denial-of-service protection limiting the number fetches for a spec in a given period of time, as a courtesy to developers.
Parse the gadget XML.
1. The XML MUST conform to the extended gadget spec XSD . This XSD represents all valid elements and attributes accepted by the legacy iGoogle gadget server (gmodules.com). However, some of these fields will be phased out. The canonical gadget spec XSD contains the subset of attributes that must be parsed and interpreted by the server. In other words, the parser MUST NOT reject gadget specs conforming to the extended spec, yet need not interpret more than the canonical spec.
Identify the Locale object corresponding to the request, and fetch messages specified by it.
1. The element is selected from the gadget XML by matching to the declared language and/or country. If an element matches both, use it. Otherwise select a language-only match over a country-only match. If no such match exists and an element is defined without language or country attributes defined, use it.
2. Load the message bundle specified in the matched element, if any.
  1. If the message attribute is specified, use it: load the bundle from the network using the provided URL.
  2. Otherwise use the element that is a child of , if present
Substitute "hangman variables" into supported gadget spec fields. Hangman variables are of the form _____, and are replaced with string values. There are four types of hangman variables, which are defined as follows.
1. Process message bundle into __MSG_foo__ hangman variables.
  1. The bundle MUST conform to the message bundle XSD.
  2. For each message named foo with value bar, hangman expansion __MSG_foo__ = bar.
2. For each provided User Pref with key foo and value bar, hangman expansion __UP_foo__ = bar.
3. Hangman expansion __MODULE_ID__ = .
  1. HTTP-based request servers SHOULD use the URL parameter named mid for this.
4. If a message bundle was found with language_direction = "rtl" , hangman expansions __BIDI_START_EDGE__ = "right", __BIDI_END_EDGE__ = "left", __BIDI_DIR__ = "rtl", __BIDI_REVERSE_DIR__ = "ltr". Otherwise __BIDI_START_EDGE__ = "left", __BIDI_END_EDGE__ = "right", __BIDI_DIR__ = "ltr", __BIDI_REVERSE_DIR__ = "rtl".
5. Perform substitutions of each hangman expansion on all fields which get displayed to users. This currently includes, but is not limited to, the following fields:
  1. Module.ModulePrefs attributes, Module.Content@href, Module.Content, UserPref@display_name, UserPref@default_value, and UserPref.EnumValue@display_value
Process gadget features, specified as or in the gadget XML.
1. If one or more unsupported features are specified in a block, the server MUST emit a standard error message indicating that it cannot satisfy the rendering request.
  1. The message SHOULD list all requested features that could not be used because the container does not support them.
  2. The message MAY provide contact information for the gadget server administrator, along with a mechanism to request adding support for the feature.
2. The server MUST support the following core JavaScript API, which applies to features:
  gadgets.util.hasFeature(featureName)
  As indicated in the JsDoc, this method returns true if the server is able to satisfy featureName, false otherwise. Gadget developers can use this functionality to enhance their gadgets if features are available without disabling their gadget if the features are missing.
3. The server MUST conform to the requirements specified by each requested feature in order to claim support for it. The particulars of this vary per feature, but include injecting JavaScript APIs into rendered output or the type URL-included libraries and manipulating type HTML content to support extended syntax.
Output gadget content.
1. For type HTML gadgets, the order is as follows:
  1. Standard HTML header, opening tag and tag. information is optional. Gadgets run in browser quirks mode.
  2. Core gadgets JavaScript libraries, as specified here.
  3. Feature library initialization code, if needed.
  4. Processed gadget content, the result of steps #4 and possibly #5.
  5. A single call to gadgets.util.runOnLoadHandlers();
  6. Standard HTML closing tags.
2. For type URL gadgets, the provided URL MUST have the following parameters added to it:
  1. up_= for each provided user preference.
  2. lang= and country=, from the provided request Locale.
  3. libs=, where is a comma-separated list of URL path fragments relative to the server's JavaScript request handler path. These fragments are added to the JS path by the type URL target to load gadget API JavaScript into its execution context.
    1. The server SHOULD consolidate all libs into a single request to minimize the number of HTTP requests made by the client browser.

Gadget Metadata Request

This API tells a gadget container how to render a gadget on a page, and gives it the container-side JavaScript it needs to support the gadget's features.

Context (container-side JavaScript):

It is often the case that the gadget container's support is required to satisfy gadget API requests due to the browser security model. For example, the dynamic-height feature allows a gadget to request that it be resized to fit its contents, particularly when it is in an IFRAME. Most often, the gadget is rendered on a different domain than its container, so it cannot modify the height itself. Rather, it makes an inter-domain procedure call to its container requesting that it be resized. This communication occurs through the core gadgets.rpc.* APIs, which use browser-specific techniques to pass messages between frames on different domains located in the same browser context. The container needs to be able to receive this request and respond accordingly - this is the function of the container-side JavaScript.

Sanitized Gadget Content Note:

Rather than a Gadget Rendering Request URL, the Gadget Metadata Request API may return a version of the gadget output sanitized for security. The Caja project provides this sanitization by rewriting HTML, CSS, and JavaScript through static analysis techniques to preclude unsafe DOM or cookie access to the gadget content, while maintaining expressiveness nearly equivalent to content running simply inside an IFRAME. This functionality is still experimental, however, so is not included in the core spec at this time.

Inputs:

Same as for the Gadget Rendering Request.

Outputs:

URL to a Gadget Rendering Request for the requested gadget.
Reference(s) to container JavaScript needed to satisfy needed gadget API requests.

Process:

Load and parse gadget XML as described in Gadget Rendering Request steps #1 and #2.
Process gadget features, specified as or in the gadget XML.
1. Emit an error message if a required feature is not supported.
2. Otherwise, for each requested, supported feature with container-side JavaScript, enqueue this JavaScript to return to the client.
Construct URL to the corresponding Gadget Rendering Request, generated as follows:
1. Initialize URL using a static prefix.
2. Add parameter url=, where is URL of the requested gadget spec.
3. Add user prefs parameters as described in Gadget Rendering Request 6.ii.a.
4. Add locale parameters as described in Gadget Rendering Request 6.ii.b.
Return the enqueued container-side JavaScript and Gadget Rendering Request URL.

JavaScript Request

In order to satisfy type URL gadgets' JavaScript library loading requests, a server MUST provide an HTTP service for retrieving core and feature-linked JavaScript.

Input:

Fixed path, computed by a static base URL combined with a path fragment as described in Gadget Rendering Request part 6.ii.c.

Output:

JavaScript code corresponding to the API(s) required by the type URL gadget.

Core JavaScript API

All core APIs MUST be provided by the server in such a way as to conform to the Gadgets API Reference. Gadget developers should not have to request any of these features, as the container should provide them all automatically.

File	JsDoc
prefs.js	Documentation
io.js	Documentation
json.js	Documentation
util.js	Documentation

Highly Recommended Features

The vast majority of gadgets include one or more features. The more features a gadget server supports, the more gadgets it supports. Thus, each gadget server strongly SHOULD support as many such features as possible, especially the following features that have high usage rates. Each supplies a JavaScript API to be made available to the gadget. Features are listed with their name(s) and their associated API:

Feature	File	JsDoc
tabs	tabs.js	Documentation
minimessage	minimessage.js	Documentation
flash	flash.js	Documentation
rpc	rpc.js	Documentation
views	views.js	Documentation
skins	skins.js	Documentation
dynamic-height	dynamic-height.js	Documentation
settitle	settitle.js	Documentation

Overview

OpenSocial is a set of APIs for building social applications that run on the web. OpenSocial's goal is to make more apps available to more users, by providing a common API that can be used in many different contexts. Developers can create applications, using standard JavaScript and HTML, that run on social websites that have implemented the OpenSocial APIs. These websites, known as OpenSocial containers, allow developers to access their social information; in return they receive a large suite of applications for their users.

The OpenSocial APIs expose methods for accessing information about people, their friends, and their data, within the context of a container. This means that when running an application on Orkut, you'll be interacting with your Orkut friends, while running the same application on MySpace lets you interact with your MySpace friends. For more information on the types of information exposed by the OpenSocial API, see the Key concepts section.

This document describes the concepts and principles behind the OpenSocial JavaScript APIs. It is a complement to the JavaScript API Reference, which covers the details of specific methods and parameters. This document does not describe any particular implementation of the API.

Compliance

So what does it mean to be an OpenSocial container? In a practical sense, it means that a site can run any application built on the OpenSocial APIs. This implies a number of things:

The container must implement all methods in the JavaScript API Reference.
Methods may return the error code opensocial.ResponseItem.NOT_IMPLEMENTED for a specific method if the container does not support a specific request. No additional public methods or classes may be put on the OpenSocial namespace at any level. The required methods and fields are defined in the following JavaScript files:
The container must only use the specified extensibility mechanisms for any container-specific extensions.
Extra person, activity or other object fields should be defined in an enum under the container's namespace, and the environment should allow applications to discover these fields. For example, if the field orkut.PersonField.SPECIAL_FIELD is defined as "orkut.specialPersonField", then opensocial.getEnvironment().supportsField("person", "orkut.specialPersonField")opensocial.getEnvironment().supportsField(opensocial.Environment.ObjectType.PERSON, orkut.PersonField.SPECIAL_FIELD) should both return true. and

Any extra data request types should be available with a namespaced call like myspace.newFetchAlbumRequest. The gadget would then use batchRequest.add(myspace.newFetchAlbumRequest(...)).

Any extra objects may be added to the container's own namespace. These objects can be returned from person.getField or other similar requests.
The container must satisfy the Gadgets API Specification.
In short, this requires handling three types of requests: the Gadget Rendering Request, the Gadget Metadata Request, and the JavaScript Request.

For the JavaScript APIs, you must implement the gadgets core JavaScript API, as per the documentation.
The container must support the RESTful API Specification.
Containers must provide both JSON and AtomPub represenatations of data. OpenSocial uses HTTP GET to retrieve, PUT to update in place, POST to create new, and DELETE to remove. POST operates on collections and creates new activities, persons, or app data within those collections.

Background

The rest of this document provides background information only and is not part of the OpenSocial specification. OpenSocial container compliance is determined solely based on the previous section, Compliance.

Key concepts

Social applications revolve around people and their relationships. OpenSocial provides a standard way for websites to expose their social graph and more. Seeing the activities of other people helps you stay up to date with your friends, and allows everything from resumes to videos to spread virally through the graph. OpenSocial also provides a way for application data to persist on a social networking site, as well as specifying the different ways that an application can be viewed within an OpenSocial container. Finally, OpenSocial defines an API through which users of web services, mobile devices, and desktop applications may interact with social data

People

It's people! Social graphs are made out of people! People are a fundamental part of social networking software and the OpenSocial API. The Person object provides access to a user's information. Part of this information is stored in the user's profile and, depending on the site, can include anything from "favorite TV shows" to "5 things you'll find in my bedroom." The other important user information is the set of connections they have in the social graph and this is covered in the Relationships section.

There are two Person objects that can be requested directly—the VIEWER and the OWNER. To understand the distinction, imagine you're checking out a coworker's profile on Orkut. In this case, you are the VIEWER and your coworker is the OWNER. It's also common to view your own profile, in which case you are both the VIEWER and the OWNER, and some applications may choose to handle this case differently. OpenSocial also provides for the case of anonymous viewing, where the gadget will not be able to access the VIEWER's information.

The API Reference contains more detailed information about the Person class.

A note about user IDs

One of the pieces of data that is always returned with a Person object is the user's ID. The user ID must be alphanumeric (A-Za-z0-9) and must uniquely identify the user in a container. This standardization is intended to allow for prefixing IDs with a domain name and separator to create globally unique IDs (e.g. "orkut.com:34KJDCSKJN2HHF0DW20394"). Note that there will likely be a size limit placed on user IDs to help manage storing IDs in a database.

Relationships

The ability to create relationships is what turns a multi-user application into social software. Being able to share information and interact with friends changes the dynamic of user experience—you're engaging with people, not software.

There are two representations of sets of people available to OpenSocial—VIEWER_FRIENDS and OWNER_FRIENDS. In the case where you're checking out a coworker's profile, requesting VIEWER_FRIENDS will return the set of users that are friends with you, while requesting OWNER_FRIENDS will return the set of users that are friends with your coworker. Logically, if you're viewing your own profile, VIEWER_FRIENDS and OWNER_FRIENDS will be the same set of users. Also, if the container support anonymous profile browsing, then the application won't be able to access the set of VIEWER_FRIENDS.

Note that OpenSocial makes no assumptions about the relationship between VIEWER and OWNER. The VIEWER and OWNER could be friends, but if you're looking at a stranger's profile, there's no relationship between you, the VIEWER, and them, the OWNER.

When an application wants to interact with or display data for "friends of friends", the OpenSocial specification supports extending a query for VIEWER_FRIENDS or OWNER_FRIENDS by a NETWORK_DISTANCE parameter. Containers can optionally support "friends of friends" queries, "friends of friends of friends" queries, and so on.

Activities

Since we can't be online all the time, it helps to have a record of what our friends have been up to, so long as your friends want to share that record. Seeing how other people are interacting with a social application also allows you to learn new features and uses of the application, so activity streams are one of the major drivers for organic growth of applications.

OpenSocial exposes activity streams, which are a collection of actions a user has taken in the context of a given container. These activities can include interaction with the container itself, such as updating your profile or installing a new gadget, or interaction with an OpenSocial application, such as sending your friend a virtual gift or setting a new high score in a game.

Activity templates allow application developers to define messages with placeholders for pieces of applicaton or user data. This separation of data and presentation allows multiple activities to be combined into activity summaries—consolidated bundles of activities that let users know what their friends are up to without having to wade through a flood of messages.

The API Reference contains more detailed information about the Activity class.

Persistence

Applications can provide a richer user experience if they can save their state between sessions. OpenSocial defines a data store that applications can use to read and write user-specific data. This data store can be read by anyone who can see the gadget, but only the VIEWER's user-scoped data is writable.

Clearly this free data store could be abused, so containers may implement quotas or rate limits to preserve their disk space. However, OpenSocial does not currently define these policies.

The keys that developers specify to index this data must only contain alphanumeric (A-Za-z0-9) characters, underscore(_), dot(.) or dash(-).

Since application data is often created based on user inputs, OpenSocial containers perform automatic HTML escaping of all application data. However, developers have the option of turning off this escaping by setting the escapeType parameter on newFetchPersonAppData and getFieldPersistence section of the OpenSocial Developer's Guide for more details. calls. See the

Views

Containers can support several different locations where gadgets may be rendered. Such locations are formally called views (formerly Surfaces in earlier versions of the API). All gadgets (not just OpenSocial applications) have become view-aware. A gadget can ask which view it is currently being rendered on, as well as what views the container supports.

Containers may define their own views, but the specification defines the following standard views in the gadgets.views.ViewType object:

Profile - A gadget on the profile view is rendered alongside other applications in the user's profile, so it will be smaller and can't support passing URL parameters from the container to the gadget.
Canvas - A gadget in the canvas view is rendered by itself, so it will have lots of real estate and URL parameters passed to the container page can be forwarded to the gadget.
Home - A gadget in the home view is rendered on a private "homepage" where the OWNER is always the same person as the VIEWER. There may be multiple gadgets present in this view.
Preview - A gadget in the preview view does not have access to either the OWNER or VIEWER objects. This view is intended to provide "test drive" functionality so that gadgets can demonstrate functionality for potential users

In addition to requesting data about the current view, gadgets may also request that the container navigate the user to another view. Imagine a news gadget that normally runs in a small profile view that changes to a full canvas view if the user clicks on a headline to get more information. Gadgets can choose the best rendering context for their current state.

When navigating to another view, a gadget may also request a change in OWNER. For example, imagine an application that places a survey on the user's profile view. This application may want to provide links to the user's friends' profiles, so that the user can see which surveys his or her friends are currently displaying.

Containers can also specify a template that applications can use to generate URLs to a specific application canvas and user. Such URLs can be shared via activity updates and printed on external websites, so linking to applications from external sources is easy.

Lifecycle events

Certain types of events may occur during the lifecycle of a gadget that are desirable yet difficult for developers to track. Such events include installs, uninstalls, rate limiting, directory listing changes, and blacklist/whitelist notifications. The gadget specification allows developers to specify URLs that the container will POST event data to when certain events are triggered.

By tracking the data POSTed to the specified URL, gadget developers can accurately track number of installs, remove database entries upon uninstall, and get automatic notifications if their gadget exceeds a quota or is marked as spam by the container. A container may optionally support sending lifecycle events to an application developer's site by sending relevant query parameters to a URL endpoint. To receive these events, you can place one or more Link tags in your application's XML. Each Link tag has a rel and href attribute. The href attribute denotes the endpoint where event pings are sent. If the rel attribute is "opensocialevent", then all events are sent to that endpoint. If the rel attribute matches "opensocialevent.TYPE", then events of TYPE are sent to that endpoint. An optional method attribute can be set to POST or GET to specify how the request should be sent. The default is GET.

Here are some examples:

The following query parameter will always be sent to the endpoint:

eventtype - the event type

Note that these requests will always be signed by the container, and your server should check that the signature is valid by using the OAuth algorithm for signature verification.

Most events will have information about one or more OpenSocial ID values. These id values are passed as one or more id attributes. Note that a single ping may aggregate a number of events by specifying many id values.

The following event types are defined. Container custom events should be designated with a domain name prefix to avoid namespace collisions.

Event	Description
`addapp`	`id` attribute identifies users who have installed the app. `from` attribute designates how the user added this app. Values are "invite", "gallery", "external". Optional.
`removeapp`	`id` attribute identifies users who have removed the app.
`app`	`action` attribute designates the action performed on the app. Values are "enabled", "disabled", and "approved". `reason` attribute identifies the reason for the action. Values are "policy", "quota", "maintenance". Optional.
`invite`	`id` attribute indicates users who have been invited. `from_id` attribute indicates the user who sent the invitation.

RESTful API

When is an OpenSocial application not just a gadget? When it has a backend server that talks directly to the container, of course! The OpenSocial RESTful API specification defines a way for servers, mobile devices, and desktop computers to interact with an OpenSocial container without requiring JavaScript or direct user interaction. Applications can update activities, send messages, and fetch OpenSocial data in the background using either an AtomPub or JSON REST service.

API Patterns

This section describes several common patterns in the OpenSocial API. Understanding these concepts should help you wade through the API Reference.

Making Requests

The OpenSocial API queries the container asynchronously for most of its calls, which is why most OpenSocial methods don't directly return data, but rather allow you to specify a callback that will be executed when the server's response is ready. Of course, making lots of asynchronous requests isn't always ideal, so the API allows for batch requests to allow developers to ask for many pieces of information at once. A developer can create an opensocial.DataRequest and add several individual request objects to it. Upon receiving the DataRequest, the container can process each request optimally and return the results of each operation as a batched result object. Containers must preserve the semantics of executing requests in serial order, though. A request that contains a write and then a read must return the newly written data, while a request that contains a read and then a write must return the data that was present before the write took place.

Capabilities Discovery

The gadgets and OpenSocial specifications determine the common APIs that all containers will support, but there are cases where a certain method or a profile field will be offered as an extension in some containers. To help developers write gadgets that can take advantage of these extensions, yet degrade gracefully in their absence, these APIs include gadgets.util.hasFeature and opensocial.Environment.supportsField methods to query the container at runtime and determine which features are available.

Of course, OpenSocial has compliance rules that must be adhered to by containers wishing to extend the API.

Action Requests and Permissions

There are cases where a gadget may wish to perform an action that needs approval by the user or mediation by the container. OpenSocial supports "request" features that allow the container to decide how to handle the interaction with the user. Functions like opensocial.requestCreateActivity, and opensocial.requestPermission allow the container to inject its own policy decisions into the gadget execution flow and notify the gadget of the result. Under this specification, it is equally valid for a container to defer to the user, always approve, always deny, or use any other method to determine responses to request calls. Additionally, this allows the container to enforce UI flows in a safe and integrated way.

Profile fields

User profiles are different from container to container. On consumer-oriented containers, applications may be interested in a user's idea of a perfect first date, but this may not be appropriate for business-oriented containers. This is why OpenSocial allows containers to extend Person objects with arbitrary fields—to match the feel and uses of each container site. Profile fields are therefore obtained through Person.getField calls rather than having hardcoded getters and setters for each property. See the Compliance and Capabilities discovery sections for more information about how containers can add additional data to the API, and how applications can determine which fields are available for use.

Fetching content from remote servers

Gadgets can use the gadgets.io.makeRequest method to pull HTML, JSON, and ATOM data from remote servers.

The gadgets.io.makeRequest call can also be used to pass data from gadgets back to application servers in a way that cannot be spoofed. The container is expected to mediate communications from the gadget to the application server (e.g. ilike.com). Therefore trusted content fetching has two main steps: [1] the gadget contacts the container, and [2] the container contacts the application server.

When the gadget contacts the container, the container needs to be able to validate any parameters it knows about: the viewer id, the owner id (if known), and the application id. Validating these parameters is an implementation detail that may vary between containers. For example, Orkut uses an encrypted token passed in the document fragment of the gadget URL.
When the container contacts the app server, the app server needs to be able to validate that the parameters really came from the container (and were not forged by some other entity). OpenSocial uses OAuth's parameter signing algorithm Note that most of the functionality in the OAuth spec, including token exchange, is not required; OpenSocial only plans to use the parameter signing piece of the spec (including timestamp and nonce). OpenSocial permits the HMAC-SHA1 method (except that the key is a shared secret between container and app, not a concatenation of tokens as specified in section 9.2 of the OAuth spec) and the RSA-SHA1 method. HMAC-SHA1 is faster and easier to implement, but it requires more coordination than RSA-SHA1.

If a user needs to fetch data from a protected resource, the makeRequest call supports full OAuth authentication via a container proxy. Gadgets can now access protected web services without requiring the user to share their password with the gadget, allowing for sophisticated and secure integration with web APIs that support OAuth.

Caching

When gadgets call makeRequest, the calls may be cached. This is great for reducing the load that your servers have to be able to handle, since OpenSocial applications may have millions of users, each requesting data from your server. However, this caching of makeRequest calls can wind up displaying old data to users if the content returned by your server changes often.

If you expect the content at the URL you are fetching to change frequently, you may want to work around the automatic caching with a more finely grained method. You can define a refreshInterval parameter, which should be specified as the number of seconds that should elapse before the makeRequest call requests content from your servers again. For example, specifying 10 will mean that the container would only query the supplied URL once every 10 seconds, no matter how many people are using the application. Specifying 0 will query the server each time the function is called, effectively bypassing the cached version.

Using the API reference

The JavaScript API reference is a complete listing of the methods and fields that OpenSocial requires, but some conventions are not immediately understandable. This section will explain some of the assumptions used in generating the API reference.

Virtual typing

JavaScript is not strongly typed, but in order to make the developer experience consistent, the APIs are defined as if JavaScript supported strongly typed data. Cases where a parameter or return value are defined as a certain type in the documentation indicate that this is the only type that can be provided or accepted for OpenSocial compatibility. Any exceptions to this rule will be directly documented in the reference.

As an example, the getErrorCode method is defined in the JavaScript API reference as returning a value of the type opensocial.ResponseItem.Error, although in practice this value will be a normal JavaScript string. However, developers can be assured that in a compliant container, the string returned is one of the opensocial.ResponseItem.Error fields.

Enumerations figure heavily into the OpenSocial API. Developers are permitted to reference an enum or use the enum's value directly. To avoid collisions related to extensibility, containers must ensure that the names and values for an enum match the ones defined in the spec and that custom values are namespaced appropriately. For example, an Orkut custom field named MY_PERSON_FIELD could have a name of Orkut.PersonField.MY_PERSON_FIELD and a value of "Orkut.myPersonField". See the Compliance and Capabilities discovery sections for more information.

Parameter maps

Many of the OpenSocial methods have a large number of optional parameters. JavaScript's lack of strong typing makes overloading difficult at best, and having functions with a large amount of parameters can be difficult to manage (especially for developers not using a nice IDE). OpenSocial uses the "object bag" approach to optional parameters—many methods take an additional opt_params parameter that takes the form of a map of parameter names to their corresponding values.

For each method which uses an opt_params map the spec defines all of the valid fields that will be read out of the map by the container. These fields have defined types as discussed in the Virtual typing section. If a developer adds an invalid value for a field in the map the container should return the BAD_REQUEST error code.

courtsy: apache.org and goole.com

Load balancing with PHP and MySql

Your enterprise portal or ERP site's server might have overloaded with huge records of your users and admin data. And you might have tried lot of options to over come this problem raised from your success eventually.
But there is a ultimate solution after optimize the architecture and code. It is Load Balancing.
Let take a look what is load balancing at a glance. Balance the overhead or load from one server to distribute into several replicated servers. So that application can manage the load very efficiently. Clustering and replication sometimes synonymous but it has also some basic and technological differences.
First we will see what is replication and how to setup in respect of MySql.
There are a number of different methods for setting up replication, and the exact method that you use will depend on how you are setting up replication, and whether you already have data within your master database.
There are some generic tasks which may be required for all replication setups:
You may want to create a separate user that will be used by your slaves to authenticate with the master to read the binary log for replication. The step is optional.
You must configure the master to support the binary log and configure a unique ID.
You must configure a unique ID for each slave that you want to connect to the Master.
Before starting a data snapshot or the replication process, you should record the position of the binary log on the master. You will need this information when configuring the slave so that the slave knows where within the binary log to start executing events.
If you already have data on your Master and you want to synchronize your slave with this base data, then you will need to create a data snapshot of your database. You can create a snapshot using mysqldump by copying the data files directly.
You will need to configure the slave with the Master settings, such as the hostname, login credentials and binary log name and positions.
Once you have configured the basic options, you will need to follow the instructions for your replication setup. A number of alternatives are provided:
If you are setting up a new MySQL master and one or more slaves, then you need only set up the configuration, as you have no data to exchange. For guidance on setting up replication in this situation,
If you are already running a MySQL server, and therefore already have data that will need to be transferred to your slaves before replication starts, have not previously configured the binary log and are able to shut down your MySQL server for a short period during the process,
If you are setting up additional slaves to an existing replication environment then you can set up the slaves without affecting the master.
Creating a User for Replication
Each Slave must connect to the Master using a standard username and password. The user that you use for this operation can be any user, providing they have been granted the REPLICATION SLAVE privilege.
You do not need to create a specific user for replication. However, you should be aware that the username and password will be stored in plain text within the master.info file. Therefore you may want to create a user that only has privileges for the replication process.
To create a user or grant an existing user the privileges required for replication use the GRANT statement. If you create a user solely for the purposes of replication then that user only needs the REPLICATION SLAVE privilege. For example, to create a user, repl, that allows all hosts within the domain mydomain.com to connect for replication:

mysql> GRANT REPLICATION SLAVE ON *.*
-> TO 'repl'@'%.mydomain.com' IDENTIFIED BY 'slavepass';

You may wish to create a different user for each slave, or use the same user for each slave that needs to connect. As long as each user that you want to use for the replication process has the REPLICATION SLAVE privilege you can create as many users as you require.
Setting the Replication Master Configuration
For replication to work you must enable binary logging on the master. If binary logging is not enabled, replication will not be possible as it is the binary log that is used to exchange data between the master and slaves.
Each server within a replication group must have a unique server-id. The server-id is used to identify individual servers within the group, and must be positive integer between 1 and (2³²)-1). How you organize and select the numbers is entirely up to you.
To configure both these options you will need to shut down your MySQL server and edit the configuration of the my.cnf or my.ini file.
You will need to add the following options to the configuration file within the [mysqld] section. If these options already exist, but are commented out, uncomment the options and alter them according to your needs. For example, to enable binary logging, using a log filename prefix of mysql-bin, and setting a server ID of 1:

[mysqld]
log-bin=mysql-bin
server-id=1
Note
For the greatest possible durability and consistency in a           replication setup using InnoDB with           transactions, you should use           innodb_flush_log_at_trx_commit=1 and           sync_binlog=1 in the master           my.cnf file.       
Note

Ensure that the skip-networking option has not been enabled on your replication master. If networking has been disabled, then your slave will not able to communicate with the master and replication will fail.

Setting the Replication Slave Configuration

The only option you must configure on the slave is to set the unique server ID. If this option is not already set, or the current value conflicts with the value that you have chosen for the master server, then you should shut down your slave server, and edit the configuration to specify the server id. For example:

[mysqld]
server-id=2

If you are setting up multiple slaves, each one must have a unique server-id value that differs from that of the master and from each of the other slaves. Think of server-id values as something similar to IP addresses: These IDs uniquely identify each server instance in the community of replication partners.

If you do not specify a server-id value, it is set to 1 if you have not defined master-host; otherwise it is set to 2. Note that in the case of server-id omission, a master refuses connections from all slaves, and a slave refuses to connect to a master. Thus, omitting server-id is good only for backup with a binary log.

You do not have to enable binary logging on the slave for replication to be enabled. However, if you enable binary logging on the slave then you can use the binary log for data backups and crash recovery on the slave, and also use the slave as part of a more complex replication topology.

Obtaining the Master Replication Information

To configure replication on the slave you must determine the masters current point within the master binary log. You will need this information so that when the slave starts the replication process, it is able to start processing events from the binary log at the correct point.

If you have existing data on your master that you want to synchronize on your slaves before starting the replication process, then you must stop processing statements on the master, obtain the current position, and then dump the data, before allowing the master to continue executing statements. If you do not stop the execution of statements then the data dump, the master status information that you use will not match and you will end up with inconsistent or corrupted databases on the slaves.

To get the master status information, follow these steps:

Start the command line client and flush all tables and block write statements by executing the FLUSH TABLES WITH READ LOCK statement:
```
mysql> FLUSH TABLES WITH READ LOCK;
```
For InnoDB tables, note that FLUSH TABLES WITH READ LOCK also blocks COMMIT operations.

Warning

Leave the client from which you issued the FLUSH TABLES statement running so that the read lock remains in effect. If you exit the client, the lock is released.
Use the SHOW MASTER STATUS statement to determine the current binary log name and offset on the master:
```
mysql > SHOW MASTER STATUS;
+---------------+----------+--------------+------------------+
| File          | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+---------------+----------+--------------+------------------+
| mysql-bin.003 | 73       | test         | manual,mysql     |
+---------------+----------+--------------+------------------+
```
The File column shows the name of the log and Position shows the offset within the file. In this example, the binary log file is mysql-bin.003 and the offset is 73. Record these values. You need them later when you are setting up the slave. They represent the replication coordinates at which the slave should begin processing new updates from the master.

If the master has been running previously without binary logging enabled, the log name and position values displayed by SHOW MASTER STATUS or mysqldump --master-data will be empty. In that case, the values that you need to use later when specifying the slave's log file and position are the empty string ('') and 4.

You now have the information you need to enable the slave to start reading from the binary log in the correct place to start replication.

If you have existing data that needs be to synchronized with the slave before you start replication, leave the client running so that the lock remains in place If you are setting up a brand new master and slave replication group, then you can exit the client and release the locks.

Creating a Data Snapshot Using `mysqldump`

One way to create a snapshot of the data in an existing master database is to use the mysqldump tool. Once the data dump has been completed, you then import this data into the slave before starting the replication process.

To obtain a snapshot of the data using mysqldump:

If you haven't already locked the tables on the server to prevent queries that update data from executing:

Start the command line client and flush all tables and block write statements by executing the FLUSH TABLES WITH READ LOCK statement:
```
mysql> FLUSH TABLES WITH READ LOCK;
```
Remember to use SHOW MASTER STATUS and record the binary log details for use when starting up the slave. The point in time of your snapshot and the binary log position must match.
In another session, use mysqldump to create a dump either of all the databases you want to replicate, or by selecting specific databases individually. For example:
```
shell> mysqldump --all-databases --lock-all-tables >dbdump.db
```
An alternative to using a bare dump, is to use the --master-data option, which will automatically append the CHANGE MASTER statement required on the slave to start the replication process.
```
shell> mysqldump --all-databases --master-data >dbdump.db
```

When choosing databases to include in the dump, remember that you will need to filter out databases on each slave that you do not want to include in the replication process.

You will need either to copy the dump file to the slave, or to use the file from the master when connecting remotely to the slave to import the data.

Creating a Data Snapshot Using Raw Data Files

If your database is particularly large then copying the raw data files may be more efficient than using mysqldump and importing the file on each slave.

However, using this method with tables in storage engines with complex caching or logging algorithms may not give you a perfect “in time” snapshot as cache information and logging updates may not have been applied, even if you have acquired a global read lock. How the storage engine responds to this depends on its crash recovery abilities.

For example, if you are using InnoDB tables, you should use the InnoDB Hot Backup tool to obtain a consistent snapshot. This tool records the log name and offset corresponding to the snapshot to be later used on the slave. Hot Backup is a non-free (commercial) tool that is not included in the standard MySQL distribution. See the InnoDB Hot Backup home page at http://www.innodb.com/hot-backup for detailed information.

Otherwise, you can obtain a reliable binary snapshot of InnoDB tables only after shutting down the MySQL Server.

To create a raw data snapshot of MyISAM tables you can use standard copy tools such as cp or copy, a remote copy tool such as scp or rsync an archiving tool such as zip or tar, or a file system snapshot tool such as dump, providing that your MySQL data files exist on a single filesystem. If you are only replicating certain databases then make sure you only copy those files that related to those tables. (For InnoDB, all tables in all databases are stored in a single file unless you have the innodb_file_per_table option enabled.)

You may want to specifically exclude the following files from your archive:

Files relating to the mysql database.
The master.info file.
The master's binary log files.
Any relay log files.

To get the most consistent results with a raw data snapshot you should shut down the server during the process, as below:

Acquire a read lock and get the master's status.
In a separate session, shut down the MySQL server:
```
shell> mysqladmin shutdown
```
Take a copy of the MySQL data files. Examples are shown below for common solutions - you need to choose only one of these solutions:
```
shell> tar cf /tmp/db.tar ./data
shell> zip -r /tmp/db.zip ./data
shell> rsync --recursive ./data /tmp/dbdata 
```
Start up the MySQL instance on the master.

If you are not using InnoDB tables, you can get a snapshot of the system from a master without shutting down the server as described in the following steps:

Acquire a read lock and get the master's status.
Take a copy of the MySQL data files. Examples are shown below for common solutions - you need to choose only one of these solutions:
```
shell> tar cf /tmp/db.tar ./data
shell> zip -r /tmp/db.zip ./data
shell> rsync --recursive ./data /tmp/dbdata 
```
In the client where you acquired the read lock, free the lock:
```
mysql> UNLOCK TABLES;
```

Once you have created the archive or copy of the database, you will need to copy the files to each slave before starting the slave replication process.

Setting Up Replication with New Master and Slaves

Setting up replication with a new Master and Slaves (i.e. with no existing data) is the easiest and most straightforward method for setting up replication.

You can also use this method if you are setting up new servers and have an existing dump of the databases that you want to load into your replication configuration. By loading the data onto a new master, the data will be automatically replicated to the slaves.

To set up replication between a new master and slave:

Configure the MySQL master with the necessary configuration properties.
Start up the MySQL master.
Setup a user,
Obtain the master status information.
Free the read lock:
```
mysql> UNLOCK TABLES;
```
On the slave, edit the MySQL configuration.
Start up the MySQL slave.
Execute the CHANGE MASTER command to set the master replication server configuration.

Because there is no data to load or exchange on a new server configuration you do not need to copy or import any information.

If you are setting up a new replication environment using the data from an existing database server, you will now need to run the dump file on the master. The database updates will automatically be propagated to the slaves:

shell> mysql -h master <>

Setting Up Replication with Existing Data

When setting up replication with existing data, you will need to decide how best to get the data

from the master to the slave before starting the replication service.

The basic process for setting up replication with existing data is as follows:

If you have not already configured the server-id and binary logging, you will need
to shut down your master to configure these options.
If you have to shut down your master database, then this is a good opportunity to take a
snapshot of the database. You should obtain the master status before
taking the database down, updating the configuration and taking a snapshot. For information on how to create a snapshot using raw data files,
If your server is already correctly configured, obtain the master status and then use mysqldump to take a snapshot take a raw snapshot of the live database using the guide in
With the MySQL master running, create a user to be used by the slave when connecting to the master during replication.
Update the configuration of the slave,
The next step depends on how you created the snapshot of data on the master.

If you used mysqldump:
1. Startup the slave, skipping replication by using the --skip-slave option.
2. Import the dump file:
```
shell> mysql <> 
```
If you created a snapshot using the raw data files:
1. Extract the data files into your slave data directory. For example:
```
shell> tar xvf dbdump.tar
```
  You may need to set permissions and ownership on the files to match the configuration of your slave.
2. Startup the slave, skipping replication by using the --skip-slave option.
Configure the slave with the master status information. This will tell the slave the binary log file and position within the file where replication needs to start, and configure the login credentials and hostname of the master. For more information on the statement required, see Section 18.1.1.10, “Setting the Master Configuration on the Slave”.
Start the slave threads:
```
mysql> START SLAVE; 
```

After you have performed this procedure, the slave should connect to the master and catch up on any updates that have occurred since the snapshot was taken.

If you have forgotten to set the server-id option for the master, slaves cannot connect to it.

If you have forgotten to set the server-id option for the slave, you get the following error in the slave's error log:

Warning: You should set server-id to a non-0 value if master_host is set; we will force server id to 2, but this MySQL server will not act as a slave.

You also find error messages in the slave's error log if it is not able to replicate for any other reason.

Once a slave is replicating, you can find in its data directory one file named master.info and another named relay-log.info. The slave uses these two files to keep track of how much of the master's binary log it has processed. Do not remove or edit these files unless you know exactly what you are doing and fully understand the implications. Even in that case, it is preferred that you use the CHANGE MASTER TO statement to change replication parameters. The slave will use the values specified in the statement to update the status files automatically.

Note

The content of master.info overrides some of the server options specified on the command line or in my.cnf. See Section 18.1.2, “Replication Startup Options and Variables”, for more details.

Once you have a snapshot of the master, you can use it to set up other slaves by following the slave portion of the procedure just described. You do not need to take another snapshot of the master; you can use the same one for each slave.

Introducing Additional Slaves to an Existing Replication Environment

If you want to add another slave to the existing replication configuration then you can do so without stopping the master. Instead, you duplicate the settings on the slaves.

To duplicate the slave:

Shut down the existing slave:
```
shell> mysqladmin shutdown 
```
Copy the data directory from the existing slave to the new slave. You can do this by creating an archive using tar or WinZip, or by performing a direct copy using a tool such as cp or rsync. Ensure you also copy the log files and relay log files.
Note

A common problem that is encountered when adding new replication slaves is that the new slave fails with a series of warning and error messages like these:
```
071118 16:44:10 [Warning] Neither --relay-log nor --relay-log-index were used; so
replication may break when this MySQL server acts as a slave and has his hostname
changed!! Please use '--relay-log=new_slave_hostname-relay-bin' to avoid this problem.
071118 16:44:10 [ERROR] Failed to open the relay log './old_slave_hostname-relay-bin.003525'
(relay_log_pos 22940879)
071118 16:44:10 [ERROR] Could not find target log during relay log initialization
071118 16:44:10 [ERROR] Failed to initialize the master info structure 
```
This is due to the fact that, if the --relay-log option is not specified, the relay log files contain the hostname as part of their filenames. (This is also true of the relay log index file if the --relay-log-index option is not used. See Section 18.1.2, “Replication Startup Options and Variables”, for more information about these options.)

To avoid this problem, use the same value for --relay-log on the new slave that was used on the existing slave. (If this option was not set explicitly on the existing slave, use existing_slave_hostname-relay-bin.) If this is not feasible, then copy the existing slave's relay log index file to the new slave and set the --relay-log-index option on the new slave to match what was used on the existing slave. (If this option was not set explicitly on the existing slave, use existing_slave_hostname-relay-bin.index.) Alternatively — if you have already tried to start the new slave (after following the remaining steps in this section) and have encountered errors like those described previously — then perform the following steps:
1. If you have not already done so, issue a STOP SLAVE on the new slave.
  
  If you have already started the existing slave again, issue a STOP SLAVE on the existing slave as well.
2. Copy the contents of the existing slave's relay log index file into the the new slave's relay log index file, making sure to overwrite any content already in the file.
3. Proceed with the remaining steps in this section.
Copy the master.info and relay.info files from the existing slave to the new slave. These files hold the current log positions.
Start the existing slave.
On the new slave, edit the configuration and the give the new slave a new unique server-id.
Start the new slave; the master.info file options will be used to start the replication process.
Setting the Master Configuration on the Slave

To set up the slave to communicate with the master for replication, you must tell the slave the necessary connection information. To do this, execute the following statement on the slave, replacing the option values with the actual values relevant to your system:
```
mysql> CHANGE MASTER TO
->     MASTER_HOST='master_host_name',
->     MASTER_USER='replication_user_name',
->     MASTER_PASSWORD='replication_password',
->     MASTER_LOG_FILE='recorded_log_file_name',
->     MASTER_LOG_POS=recorded_log_position;     
```
Note
Replication cannot use Unix socket files. You must be able to
connect to the master MySQL server using TCP/IP.
The following table shows the maximum allowable length for the string-valued options:

MASTER_HOST 60

MASTER_USER 16

MASTER_PASSWORD 32

MASTER_LOG_FILE 255
Replication Startup Options and Variables

This section describes the options that you can use on slave
replication servers. You can specify these options either on the
command line or in an option file.
On the master and each slave, you must use the
server-id option to establish a unique
replication ID. For each server, you should pick a unique positive
integer in the range from 1 to 2³² – 1, and each ID must be different from every other ID.
Example: server-id=3
Options that you can use on the master server for controlling
binary logging are described in Section 5.2.3, “The Binary Log”.
Important
Some slave server replication options are ignored if a
master.info file exists when the slave starts and contains a value for the option.
The following options are handled this way:
- --master-host
- --master-user
- --master-password
- --master-port
- --master-connect-retry
- --master-ssl
- --master-ssl-ca
- --master-ssl-capath
- --master-ssl-cert
- --master-ssl-cipher
- --master-ssl-key
The master.info file format in MySQL5.0 includes values corresponding to the SSL options.
In addition, the file format includes as its first line the number of lines in the file. you
upgrade an older server (before MySQL 4.1.1) to a newer version,the new server upgrades the
master.info file to the new format automatically when it starts. However, if you downgrade a
newer server to an older version, you should remove the first line manually before starting the
older server for the first time.
If no master.info file exists when the slave server starts, it uses the values for those
options that are specified in option files or on the command line. This occurs when you start
the server as a replication slave for the very first time, or when you have run RESET SLAVE
and then have shut down and restarted the slave.
If the master.info file exists when the slave server starts, the server uses its contents
and ignores any options that correspond to the values listed in the file. Thus, ifyou start the
slave server with different values of the startup options that correspond to values in the master.
info file, the different values have no effect, because the server continues to use the master.
info file. To use different values,you must either restart after removing the master.info file or
(preferably) use the CHANGE MASTER TO statement to reset the values while the slave is
running.
Suppose that you specify this option in your my.cnf file:
[mysqld]
```
master-host=some_host   
```
The first time you start the server as a replication slave, it reads and uses that
option from the my.cnf file. The server then records the value in the master.info file.
The next time you start the server, it reads the master host value from the master.info
file only and ignores the value in the option file. If you modify the my.cnf file to
specify a different master host of some_other_host, the change still has no effect.
You should use CHANGE MASTER TO instead.
MySQL Enterprise. For expert advice regarding master startup options subscribe to the
MySQL Enterprise Monitor. For more information, Because the server gives an existing master.
info file precedence over the startup options just described, you might prefer not to use
startup options for these values at all, and instead specify them by using the
CHANGE MASTER TO statement.
This example shows a more extensive use of startup options to configure a slave server:
```
[mysqld]
server-id=2
master-host=db-master.mycompany.com
master-port=3306
master-user=pertinax
master-password=freitag
master-connect-retry=60
report-host=db-slave.mycompany.
```
The following list describes startup options for controlling replication. Many of these
options can be reset while the server is running by using the CHANGE MASTER TO statement.
Others, such as the --replicate-* options, can be set only when the slave server starts.
- --log-slave-updates
  Normally, a slave does not log to its own binary log any updates that are received
- from a master server. This option tells the slave to log the updates performed by its
- SQL thread to its own binary log. For this option to have any effect, the slave must also be started with the --log-bin option to enable binary logging. --log-slave-updates is used when you want to chain replication servers. For example, you might want to set up replication servers using this arrangement:
```
A -> B -> C
```
  Here, A serves as the master for the slave B, and B serves as the master for the slave C. For this to work, B must be both a master and a slave. You must start both A and B with --log-bin to enable binary logging, and B with the --log-slave-updates option so that updates received from A are logged by B to its binary log.
- --log-warnings[=level]
  This option causes a server to print more messages to the error log about what it is doing. With respect to replication, the server generates warnings that it succeeded in reconnecting after a network/connection failure, and informs you as to how each slave thread started. This option is enabled by default; to disable it, use --skip-log-warnings. Aborted connections are not logged to the error log unless the value is greater than 1.
- --master-connect-retry=seconds
  
  The number of seconds that the slave thread sleeps before trying to reconnect to the master in case the master goes down or the connection is lost. The value in the master.info file takes precedence if it can be read. If not set, the default is 60. Connection retries are not invoked until the slave times out reading data from the master according to the value of --slave-net-timeout. The number of reconnection attempts is limited by the --master-retry-count option.
  --master-host=host_name
  The hostname or IP number of the master replication server. The value in master.info takes precedence if it can be read. If no master host is specified, the slave thread does not start.
  --master-info-file=file_name
  The name to use for the file in which the slave records information about the master. The default name is master.info in the data directory.
- --master-password=password
  The password of the account that the slave thread uses for authentication when it connects to the master. The value in the master.info file takes precedence if it can be read. If not set, an empty password is assumed.
- --master-port=port_number
  The TCP/IP port number that the master is listening on. The value in the master.info file takes precedence if it can be read. If not set, the compiled-in setting is assumed (normally 3306).
- --master-retry-count=count
  The number of times that the slave tries to connect to the master before giving up. Reconnects are attempted at intervals set by --master-connect-retry and reconnects are triggered when data reads by the slave time out according to the --slave-net-timeout option. The default value is 86400.
  
  These options are used for setting up a secure replication connection to the master server using SSL. Their meanings are the same as the corresponding --ssl, --ssl-ca, --ssl-capath, --ssl-cert, --ssl-cipher, --ssl-key options that are described in Section 5.5.7.3, “SSL Command Options”. The values in the master.info file take precedence if they can be read.
  --master-user=user_name
  The username of the account that the slave thread uses for authentication when it connects to the master. This account must have the REPLICATION SLAVE privilege. The value in the master.info file takes precedence if it can be read. If the master username is not set, the name test is assumed.
- --max-relay-log-size=size
  The size at which the server rotates relay log files automatically. For more information, see Section 18.4.2, “Replication Relay and Status Files”. The default size is 1GB.
  --read-only
  When this option is given, the server allows no updates except from users that have the SUPER privilege or (on a slave server) from updates performed by slave threads. On a slave server, this can be useful to ensure that the slave accepts updates only from its master server and not from clients. As of MySQL 5.0.16, this option does not apply to TEMPORARY tables.
  --relay-log=file_name
  The basename for the relay log. The default basename is host_name-relay-bin. The server creates relay log files in sequence by adding a numeric suffix to the basename. You can specify the option to create hostname-independent relay log names, or if your relay logs tend to be big (and you don't want to decrease max_relay_log_size) and you need to put them in some area different from the data directory, or if you want to increase speed by balancing load between disks.
  --relay-log-index=file_name
  The name to use for the relay log index file. The default name is host_name-relay-bin.index in the data directory, where host_name is the name of the slave server.
  --relay-log-info-file=file_name
  The name to use for the file in which the slave records information about the relay logs. The default name is relay-log.info in the data directory.
  --relay-log-purge={0|1}
  Disable or enable automatic purging of relay logs as soon as they are not needed any more. The default value is 1 (enabled). This is a global variable that can be changed dynamically with SET GLOBAL relay_log_purge = N.
  --relay-log-space-limit=size
  This option places an upper limit on the total size in bytes of all relay logs on the slave. A value of 0 means “no limit.” This is useful for a slave server host that has limited disk space. When the limit is reached, the I/O thread stops reading binary log events from the master server until the SQL thread has caught up and deleted some unused relay logs. Note that this limit is not absolute: There are cases where the SQL thread needs more events before it can delete relay logs. In that case, the I/O thread exceeds the limit until it becomes possible for the SQL thread to delete some relay logs, because not doing so would cause a deadlock. You should not set --relay-log-space-limit to less than twice the value of --max-relay-log-size (or --max-binlog-size if --max-relay-log-size is 0). In that case, there is a chance that the I/O thread waits for free space because --relay-log-space-limit is exceeded, but the SQL thread has no relay log to purge and is unable to satisfy the I/O thread. This forces the I/O thread to temporarily ignore --relay-log-space-limit.
  --replicate-do-db=db_name
  Tell the slave to restrict replication to statements where the default database (that is, the one selected by USE) is db_name. To specify more than one database, use this option multiple times, once for each database. Note that this does not replicate cross-database statements such as UPDATE some_db.some_table SET foo='bar' while having selected a different database or no database.
  Warning
  To specify multiple databases you must use multiple instances of this option. Because database names can contain commas, if you supply a comma separated list then the list will be treated as the name of a single database.
  An example of what does not work as you might expect: If the slave is started with --replicate-do-db=sales and you issue the following statements on the master, the UPDATE statement is not replicated:
```
USE prices;
UPDATE sales.january SET amount=amount+1000;
```
  The main reason for this “just check the default database” behavior is that it is difficult from the statement alone to know whether it should be replicated (for example, if you are using multiple-table DELETE statements or multiple-table UPDATE statements that act across multiple databases). It is also faster to check only the default database rather than all databases if there is no need.
  If you need cross-database updates to work, use --replicate-wild-do-table=db_name.% instead. See Section 18.4.3, “How Servers Evaluate Replication Rules”.
  --replicate-do-table=db_name.tbl_name
  Tell the slave thread to restrict replication to the specified table. To specify more than one table, use this option multiple times, once for each table. This works for cross-database updates, in contrast to --replicate-do-db. See Section 18.4.3, “How Servers Evaluate Replication Rules”.
  --replicate-ignore-db=db_name
  Tells the slave to not replicate any statement where the default database (that is, the one selected by USE) is db_name. To specify more than one database to ignore, use this option multiple times, once for each database. You should not use this option if you are using cross-database updates and you do not want these updates to be replicated. See Section 18.4.3, “How Servers Evaluate Replication Rules”.
  MySQL Enterprise. For expert advice regarding slave startup options subscribe to the MySQL Enterprise Monitor. For more information, see http://www.mysql.com/products/enterprise/advisors.html.
  An example of what does not work as you might expect: If the slave is started with --replicate-ignore-db=sales and you issue the following statements on the master, the UPDATE statement is replicated:
  USE prices;
```
UPDATE sales.january SET amount=amount+1000;
```
  Note
  In the preceding example the statement is replicated because --replicate-ignore-db only applies to the default database (set through the USE statement). Because the sales database was specified explicitly in the statement, the statement has not been filtered.
  If you need cross-database updates to work, use --replicate-wild-ignore-table=db_name.% instead. See Section 18.4.3, “How Servers Evaluate Replication Rules”.
- --replicate-ignore-table=db_name.tbl_name
  Tells the slave thread to not replicate any statement that updates the specified table, even if any other tables might be updated by the same statement. To specify more than one table to ignore, use this option multiple times, once for each table. This works for cross-database updates, in contrast to --replicate-ignore-db. See Section 18.4.3, “How Servers Evaluate Replication Rules”.
- --replicate-rewrite-db=from_name->to_name
  Tells the slave to translate the default database (that is, the one selected by USE) to to_name if it was from_name on the master. Only statements involving tables are affected (not statements such as CREATE DATABASE, DROP DATABASE, and ALTER DATABASE), and only if from_name is the default database on the master. This does not work for cross-database updates. To specify multiple rewrites, use this option multiple times. The server uses the first one with a from_name value that matches. The database name translation is done before the --replicate-* rules are tested.
  If you use this option on the command line and the “>” character is special to your command interpreter, quote the option value. For example:
```
shell> mysqld --replicate-rewrite-db="olddb->newdb" 
```
- --replicate-same-server-id
  To be used on slave servers. Usually you should use the default setting of 0, to prevent infinite loops caused by circular replication. If set to 1, the slave does not skip events having its own server ID. Normally, this is useful only in rare configurations. Cannot be set to 1 if --log-slave-updates is used. Note that by default the slave I/O thread does not even write binary log events to the relay log if they have the slave's server id (this optimization helps save disk usage). So if you want to use --replicate-same-server-id, be sure to start the slave with this option before you make the slave read its own events that you want the slave SQL thread to execute.
- --replicate-wild-do-table=db_name.tbl_name
  Tells the slave thread to restrict replication to statements where any of the updated tables match the specified database and table name patterns. Patterns can contain the “%” and “_” wildcard characters, which have the same meaning as for the LIKE pattern-matching operator. To specify more than one table, use this option multiple times, once for each table. This works for cross-database updates. See Section 18.4.3, “How Servers Evaluate Replication Rules”.
  Example: --replicate-wild-do-table=foo%.bar% replicates only updates that use a table where the database name starts with foo and the table name starts with bar.
  If the table name pattern is %, it matches any table name and the option also applies to database-level statements (CREATE DATABASE, DROP DATABASE, and ALTER DATABASE). For example, if you use --replicate-wild-do-table=foo%.%, database-level statements are replicated if the database name matches the pattern foo%.
  To include literal wildcard characters in the database or table name patterns, escape them with a backslash. For example, to replicate all tables of a database that is named my_own%db, but not replicate tables from the my1ownAABCdb database, you should escape the “_” and “%” characters like this: --replicate-wild-do-table=my\_own\%db. If you're using the option on the command line, you might need to double the backslashes or quote the option value, depending on your command interpreter. For example, with the bash shell, you would need to type --replicate-wild-do-table=my\\_own\\%db.
- --replicate-wild-ignore-table=db_name.tbl_name
  Tells the slave thread not to replicate a statement where any table matches the given wildcard pattern. To specify more than one table to ignore, use this option multiple times, once for each table. This works for cross-database updates. See Section 18.4.3, “How Servers Evaluate Replication Rules”.
  Example: --replicate-wild-ignore-table=foo%.bar% does not replicate updates that use a table where the database name starts with foo and the table name starts with bar.
  For information about how matching works, see the description of the --replicate-wild-do-table option. The rules for including literal wildcard characters in the option value are the same as for --replicate-wild-ignore-table as well.
- --report-host=slave_name
  The hostname or IP number of the slave to be reported to the master during slave registration. This value appears in the output of SHOW SLAVE HOSTS on the master server. Leave the value unset if you do not want the slave to register itself with the master. Note that it is not sufficient for the master to simply read the IP number of the slave from the TCP/IP socket after the slave connects. Due to NAT and other routing issues, that IP may not be valid for connecting to the slave from the master or other hosts.
- --report-password=password
  The account password of the slave to be reported to the master during slave registration. This value appears in the output of SHOW SLAVE HOSTS on the master server if the --show-slave-auth-info option is given.
- --report-port=slave_port_num
  The TCP/IP port number for connecting to the slave, to be reported to the master during slave registration. Set this only if the slave is listening on a non-default port or if you have a special tunnel from the master or other clients to the slave. If you are not sure, do not use this option.
- --report-user=user_name
  The account username of the slave to be reported to the master during slave registration. This value appears in the output of SHOW SLAVE HOSTS on the master server if the --show-slave-auth-info option is given.
- --show-slave-auth-info
  Display slave usernames and passwords in the output of SHOW SLAVE HOSTS on the master server for slaves started with the --report-user and --report-password options.
- --skip-slave-start
  Tells the slave server not to start the slave threads when the server starts. To start the threads later, use a START SLAVE statement.
- --slave_compressed_protocol={0|1}
  If this option is set to 1, use compression for the slave/master protocol if both the slave and the master support it. The default is 0 (no compression).
- --slave-load-tmpdir=file_name
  The name of the directory where the slave creates temporary files. This option is by default equal to the value of the tmpdir system variable. When the slave SQL thread replicates a LOAD DATA INFILE statement, it extracts the file to be loaded from the relay log into temporary files, and then loads these into the table. If the file loaded on the master is huge, the temporary files on the slave are huge, too. Therefore, it might be advisable to use this option to tell the slave to put temporary files in a directory located in some filesystem that has a lot of available space. In that case, the relay logs are huge as well, so you might also want to use the --relay-log option to place the relay logs in that filesystem.
  The directory specified by this option should be located in a disk-based filesystem (not a memory-based filesystem) because the temporary files used to replicate LOAD DATA INFILE must survive machine restarts. The directory also should not be one that is cleared by the operating system during the system startup process.
- --slave-net-timeout=seconds
  The number of seconds to wait for more data from the master before the slave considers the connection broken, aborts the read, and tries to reconnect. The first retry occurs immediately after the timeout. The interval between retries is controlled by the CHANGE MASTER TO statement or --master-connect-retry option and the number of reconnection attempts is limited by the --master-retry-count option. The default is 3600 seconds (one hour).
- --slave-skip-errors=[err_code1,err_code2,...|all]
  Normally, replication stops when an error occurs on the slave. This gives you the opportunity to resolve the inconsistency in the data manually. This option tells the slave SQL thread to continue replication when a statement returns any of the errors listed in the option value.
  Do not use this option unless you fully understand why you are getting errors. If there are no bugs in your replication setup and client programs, and no bugs in MySQL itself, an error that stops replication should never occur. Indiscriminate use of this option results in slaves becoming hopelessly out of synchrony with the master, with you having no idea why this has occurred.
  For error codes, you should use the numbers provided by the error message in your slave error log and in the output of SHOW SLAVE STATUS. Appendix B, Errors, Error Codes, and Common Problems, lists server error codes.
  You can also (but should not) use the very non-recommended value of all to cause the slave to ignore all error messages and keeps going regardless of what happens. Needless to say, if you use all, there are no guarantees regarding the integrity of your data. Please do not complain (or file bug reports) in this case if the slave's data is not anywhere close to what it is on the master. You have been warned.
  Examples:
```
--slave-skip-errors=1062,1053
--slave-skip-errors=all
```
Common Replication Administration Tasks

Once replication has been started it should execute without requiring much regular administration. Depending on your replication environment, you will want to check the replication status of each slave either periodically, daily, or even more frequently.

Checking Replication Status

The most common task when managing a replication process is to ensure that replication is taking place and that there have been no errors between the slave and the master.

The primary command for this is SHOW SLAVE STATUS which you must execute on each slave:

mysql> SHOW SLAVE STATUS\G
*************************** 1. row ***************************
        Slave_IO_State: Waiting for master to send event
           Master_Host: master1
           Master_User: root
           Master_Port: 3306
         Connect_Retry: 60
       Master_Log_File: mysql-bin.000004
   Read_Master_Log_Pos: 931
        Relay_Log_File: slave1-relay-bin.000056
         Relay_Log_Pos: 950
 Relay_Master_Log_File: mysql-bin.000004
      Slave_IO_Running: Yes
     Slave_SQL_Running: Yes
       Replicate_Do_DB:
   Replicate_Ignore_DB:
    Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
            Last_Errno: 0
            Last_Error:
          Skip_Counter: 0
   Exec_Master_Log_Pos: 931
       Relay_Log_Space: 1365
       Until_Condition: None
        Until_Log_File:
         Until_Log_Pos: 0
    Master_SSL_Allowed: No
    Master_SSL_CA_File:
    Master_SSL_CA_Path:
       Master_SSL_Cert:
     Master_SSL_Cipher:
        Master_SSL_Key:
 Seconds_Behind_Master: 0
1 row in set (0.01 sec)

The key fields from the status report to examine are:

Slave_IO_State — indicates the current status of the slave. See Section 7.5.5.5, “Replication Slave I/O Thread States”, and Section 7.5.5.6, “Replication Slave SQL Thread States”, for more information.
Slave_IO_Running — shows whether the IO thread for the reading the master's binary log is running.
Slave_SQL_Running — shows whether the SQL thread for the executing events in the relay log is running.
Last_Error — shows the last error registered when processing the relay log. Ideally this should be blank, indicating no errors.
Seconds_Behind_Master — shows the number of seconds that the slave SQL thread is behind processing the master binary log. A high number (or an increasing one) can indicate that the slave is unable to cope with the large number of queries from the master.
A value of 0 for Seconds_Behind_Master can usually be interpreted as meaning that the slave has caught up with the master, but there are some cases where this is not strictly true. For example, this can occur if the network connection between master and slave is broken but the slave I/O thread has not yet noticed this — that is, slave_net_timeout has not yet elapsed.
It is also possible that transient values for Seconds_Behind_Master may not reflect the situation accurately. When the slave SQL thread has caught up on I/O, Seconds_Behind_Master displays 0; but when the slave I/O thread is still queuing up a new event, Seconds_Behind_Master may show a large value until the SQL thread finishes executing the new event. This is especially likely when the events have old timestamps; in such cases, if you execute SHOW SLAVE STATUS several times in a relatively short peiod, you may see this value change back and forth repeatedly between 0 and a relatively large value.

On the master, you can check the status of slaves by examining the list of running processes. Slaves execute the Binlog Dump command:

mysql> SHOW PROCESSLIST \G;
*************************** 4. row ***************************
Id: 10
User: root
Host: slave1:58371
db: NULL
Command: Binlog Dump
Time: 777
State: Has sent all binlog to slave; waiting for binlog to be updated
Info: NULL

Because it is the slave that drives the core of the replication process, very little information is available in this report.

If you have used the --report-host option, then the SHOW SLAVE HOSTS statement will show basic information about connected slaves:

mysql> SHOW SLAVE HOSTS;
+-----------+--------+------+-------------------+-----------+
| Server_id | Host   | Port | Rpl_recovery_rank | Master_id |
+-----------+--------+------+-------------------+-----------+
|        10 | slave1 | 3306 |                 0 |         1 |
+-----------+--------+------+-------------------+-----------+
1 row in set (0.00 sec)

The output includes the ID of the slave server, the value of the --report-host option, the connecting port, master ID and the priority of the slave for receiving binary log updates.

Pausing Replication on the Slave

You can stop and start the replication of statements on the slave using the STOP SLAVE and START SLAVE commands.

To stop execution of the binary log from the master, use STOP SLAVE:

mysql> STOP SLAVE;

When execution is stopped, the slave does not read the binary log from the master (the IO_THREAD) and stops processing events from the relay log that have not yet been executed (the SQL_THREAD). You can pause either the IO or SQL threads individually by specifying the thread type. For example:

mysql> STOP SLAVE IO_THREAD;

Stopping the SQL thread can be useful if you want to perform a backup or other task on a slave that only processes events from the master. The IO thread will continue to be read from the master, but not executed, which will make it easier for the slave to catch up when you start slave operations again.

Stopping the IO thread will allow the statements in the relay log to be executed up until the point where the relay log has ceased to receive new events. Using this option can be useful when you want to pause execution to allow the slave to catch up with events from the master, when you want to perform administration on the slave but also ensure you have the latest updates to a specific point. This method can also be used to pause execution on the slave while you conduct administration on the master while ensuring that there is not a massive backlog of events to be executed when replication is started again.

To start execution again, use the START SLAVE statement:

mysql> START SLAVE;

If necessary, you can start either the IO_THREAD or SQL_THREAD threads individually.

MySQL replication works by having a master server where all the inserts, updates and deletes (basically any writing done) and one or more slave servers that polls the master server to replicate the database. You can only issue select queries to the slave server. You can also have multiple master servers but it won’t be covered here. You can follow this article to setup replication.

I’ll be using Round Robin to balance the load since I’ll be load balancing for a separate portion only where the same queries are used. This will equally split the load to each server (…almost). To do this in PHP, I wrote a very simple script that opens a socket. Once a host connects, it tells which database server to connect to and immediately terminates the connection.

#!/usr/bin/php -q
array(’db1.dbservers.net’,’someuser’,’somepass’),
1 => array(’db2.dbservers.net’,'anotheruser’,'anotherpass’),
/***** if you add another host (just follow the drift) *****/
// 2 => array(’db3.dbservers.net’,'yetanotheruser’,'yetanotherpass’)
);

// Loop forever
while(true)
{
// Accept anyone
$client = socket_accept($stream);
$key = key($hosts);
$reply = implode($hosts[$key], “|”);

// Move internal pointer to next host
if(next($hosts) === FALSE)
reset($hosts);

// Push response then kill the connection
socket_write($client, $reply);
socket_close($client);
}
?>

This script should be called from the command line and run like a daemon. Then we modify how we connect to the database. We connect to the “daemon” and catch the login information.

The code above can be improved further to check if a host is still up, give weights on the server depending on its hardware and other tools.

courtsy: mysql.com and blog.jploh.com

`MASTER_HOST`	60
`MASTER_USER`	16
`MASTER_PASSWORD`	32
`MASTER_LOG_FILE`	255

Sunday

Gadgets aplciation with PHP and Shindig server

What is Shindig?

Compliance: Gadget Server

Gadget Rendering Request

Inputs:

Outputs:

Process:

Gadget Metadata Request

Context (container-side JavaScript):

Sanitized Gadget Content Note:

Inputs:

Outputs:

Process:

JavaScript Request

Input:

Output:

Core JavaScript API

Highly Recommended Features

Overview

Compliance

Background

Key concepts

People

Relationships

Activities

Persistence

Views

Lifecycle events

RESTful API

API Patterns

Making Requests

Capabilities Discovery

Action Requests and Permissions

Profile fields

Fetching content from remote servers

Caching

Using the API reference

Virtual typing

Parameter maps

Load balancing with PHP and MySql

Setting the Replication Master Configuration

Setting the Replication Slave Configuration

Obtaining the Master Replication Information

Warning

Creating a Data Snapshot Using mysqldump

Creating a Data Snapshot Using Raw Data Files

Setting Up Replication with New Master and Slaves

Setting Up Replication with Existing Data

Note

Introducing Additional Slaves to an Existing Replication Environment

Note

Setting the Master Configuration on the Slave

Note

Replication Startup Options and Variables

Important

Pausing Replication on the Slave

Beyond Spot-Checking: Why LLM Applications Require Specialized Evaluation

Creating a Data Snapshot Using `mysqldump`