import akka.actor.{ ActorRef, ActorSystem, Props, Actor, Inbox } import scala.concurrent.duration._ // Three messages, one is object, two are case classes case object Greet case class WhoToGreet(who: String) case class Greeting(message: String) class Greeter extends Actor { var greeting = "" def receive = { case WhoToGreet(who) => greeting = s"hello, $who" case Greet => sender ! Greeting(greeting) // Send the current greeting back to the sender } } object HelloAkkaScala extends App { // Create the 'helloakka' actor system val system = ActorSystem("helloakka") // Create the 'greeter' actor val greeter = system.actorOf(Props[Greeter], "greeter") // Create an "actor-in-a-box" val inbox = Inbox.create(system) // Tell the 'greeter' to change its 'greeting' message greeter.tell(WhoToGreet("akka"), ActorRef.noSender) // Ask the 'greeter for the latest 'greeting' // Reply should go to the "actor-in-a-box" inbox.send(greeter, Greet) // Wait 5 seconds for the reply with the 'greeting' message val Greeting(message1) = inbox.receive(5.seconds) println(s"Greeting: $message1") // Change the greeting and ask for it again greeter.tell(WhoToGreet("typesafe"), ActorRef.noSender) inbox.send(greeter, Greet) val Greeting(message2) = inbox.receive(5.seconds) println(s"Greeting: $message2") val greetPrinter = system.actorOf(Props[GreetPrinter]) // after zero seconds, send a Greet message every second to the greeter with a sender of the greetPrinter system.scheduler.schedule(0.seconds, 1.second, greeter, Greet)(system.dispatcher, greetPrinter) } // prints a greeting class GreetPrinter extends Actor { def receive = { case Greeting(message) => println(message) } }
actor ! message
or actor.tell(message, sender)
.
inbox.send(sender, message)
system.scheduler.schedule(initDelay, interval, receiver,
message)(executor, sender)
actor ? message
, it means “ask”, which resturns a Future
representing a possible reply. When you implement ask
function, you have to catch the exception and send a Failure
to the sender
try { val result = operation() sender() ! result } catch { case e: Exception => sender() ! akka.actor.Status.Failure(e) throw e }
You can set a timeout period for the Future
by implicit
value implicit val timeout = Timeout(5 seconds)
or using
curry function myActor.ask(msg)(5 seconds)
.
receive
defined method in actor will be called when message
comes. Attention:, the sender()
method returns the current
sender when the method is called, DO NOT use this function
in callbacks since the sender method may returns wrong sender
(deadletter in many cases) when the callback is actually
invoked.
inbox.receive
context.setReceiveTimeout(30 milliseconds)
. Then you can
catch this event by receiving message case ReceiveTimeout
.
target forward message
, it will keep the sender unchanged. A
send msg to B, B forward msg to C, the sender for C is still A
instead of B
akka.actor.PoisonPill
message will stop the
actor. gracefulStop
is useful to wait for termination or
compose ordered termination of several actors. It is a pattern,
not precoded, it sends a user defined message to the actor and
returns a Future
, you can Await
this future for a duration.
actor ! Kill
will also stop this actor, but it causes the actor
to throw a ActorKilledException
. This actor will suspend and
like the supervisor to decide how to handle the failure.
context.become
method in Actor can setting the receive
function, hence the behavior of Actor is changed. unbecome
lets
the actor returns to previews behavior.
You can use stash
and unstash
to save mails into a queue in
one behavior and get them back in another behavior. Of course,
you have to mix the trait Stash
to be able to use it.
PartialFunction#orElse
to chain the receive
function of
the extended actor.
Three init patterns
Example code from document.
Counter
, increment and save count to storage, return current
count number
Storage
, simulate database, store (String, Long) pairs.
CounterService
, override supervisorStrategy by
OneForOneStrategy
override val supervisorStrategy = OneForOneStrategy(maxNrOfRetries = 3, withinTimeRange = 5.seconds) { case _: Storage.StorageException => Restart }
init with actor refererences to the wrapper (watcher) of storage, tell counter (when initialized?) to use the storage.
forward message (increment, getCurrentCount) to counter if it
exists; save sender->msg
pair to log if counter is not
available.
if storage is terminated, tell counter that storage is not available, schedule task to initStorage again (reconnect).
Worker
, stop when receiving message from CounterService. init
CounterService
actor.
Listener
, shutdown the system while timeout. Listen
Progress
, which is a case class of Worker
.
CounterService
to increment count and pipe the message to listener.
To run cluster, you have to first creat a configuration file that specify the URI, actors, etc.
akka { actor { provider = "akka.cluster.ClusterActorRefProvider" } remote { log-remote-lifecycle-events = off netty.tcp { hostname = "127.0.0.1" port = 0 } } cluster { seed-nodes = [ "akka.tcp://ClusterSystem@127.0.0.1:2551", "akka.tcp://ClusterSystem@127.0.0.1:2552"] auto-down-unreachable-after = 10s } }
package sample.cluster.simple import com.typesafe.config.ConfigFactory import akka.actor.ActorSystem import akka.actor.Props object SimpleClusterApp { def main(args: Array[String]): Unit = { if (args.isEmpty) startup(Seq("2551", "2552", "0")) else startup(args) } def startup(ports: Seq[String]): Unit = { ports foreach { port => // Override the configuration of the port val config = ConfigFactory.parseString("akka.remote.netty.tcp.port=" + port). withFallback(ConfigFactory.load()) // Create an Akka system val system = ActorSystem("ClusterSystem", config) // One system on one port? // Create an actor that handles cluster domain events system.actorOf(Props[SimpleClusterListener], name = "clusterListener") } } } class SimpleClusterListener extends Actor with ActorLogging { val cluster = Cluster(context.system) // subscribe to cluster changes, re-subscribe when restart override def preStart(): Unit = { //#subscribe cluster.subscribe(self, initialStateMode = InitialStateAsEvents, classOf[MemberEvent], classOf[UnreachableMember]) //#subscribe } override def postStop(): Unit = cluster.unsubscribe(self) def receive = { case MemberUp(member) => log.info("Member is Up: {}", member.address) case UnreachableMember(member) => log.info("Member detected as unreachable: {}", member) case MemberRemoved(member, previousStatus) => log.info("Member is Removed: {} after {}", member.address, previousStatus) case _: MemberEvent => // ignore } }
Log when starting the second seed node: [info] [INFO] [03/10/2015 23:57:27.799] [ClusterSystem-akka.actor.default-dispatcher-14] [Cluster(akka://ClusterSystem)] Cluster Node [akka.tcp://ClusterSystem@127.0.0.1:2551] - Node [akka.tcp://ClusterSystem@127.0.0.1:2552] is JOINING, roles []
This code defines a cluster (集群), each node runs as dependennt
processes (different JVMs) since they all run on my local
machine. A cluster is indentified by akka://CllusterSystem
,
while each node is indentified by IP address and port number. The
IP address seems like to be defined by application.config.
seed-nodes
figure
out where to join the cluster to current node. The seed nodes can
also be configured by command line options when starting the JVM
using the following syntax:
-Dakka.cluster.seed-nodes.0=akka.tcp://ClusterSystem@host1:2552 -Dakka.cluster.seed-nodes.1=akka.tcp://ClusterSystem@host1:2552
Starting seed notes can be async and not all the seed nodes are necessary to run. But the first node must be started when starting a cluster.
It is also possible to join the cluster manually via JMX or
Command Line Management or via programatic method with
Cluster.get(system).join(address)
or
Cluster(system).joinSeedNodes
.
hostname
in remote
in configuration specify the IP of
current node. Question: /what if I specify the IP to another
machine? The application should be deploy to another machine
before running? What protocole does it uses?/
Frontend-backend pattern (not those in Group router):
receive
method for
the case of Terminated(a)
.
akka.cluster.roles
or in start script as a system property
or environment variable.
Number of members restriction:
akka.cluster.min-nr-of-members = 3
, the leader will not
change the status of nodes to ‘Up’ until this number of
members reached. Finer configuration:
#+beign_src yaml akka.cluster.role { frontend.min-nr-of-members = 1 backend.min-nr-of-members = 2 }
#+end_src Define callback to be invoked when the current member status is changed to ‘Up’
Cluster(system) registerOnMemberUp { system.actorOf(Props(classOf[FactorialFrontend], upToN, true), name = "factorialFrontend") }
Router is one that receive and send tasks to actors, while the ref to actors are called routees.
在示例代码中,每一个node上面都有一个StatsService,和一个 StatsWorker. StatsService里面包含了指向StatsWorker的router并且负 责接受和转发任务. 每次Client将一个任务传给Cluster时,Cluster中某 个一node上的StatsService会接收并分发任务,因为分发任务是依靠 router来实现的,所以任务会被分发到不同node上的StatsWorker上去.这 个例子中,每个node上的Service和Worker都是显式创建的.
Group example: Server side
akka.actor.deployment {
/statsService/workerRouter {
router = consistent-hashing-group
// specify the router, you have also other options like round-robin-pool
nr-of-instances = 100
routees.paths = ["/user/statsWorker"]
cluster {
enabled = on
allow-local-routees = on
use-role = compute
}
}
}
With this configuration, relative actor paths defined in
routees.paths
are used for selecting actors to send the message
by the router.
在创建System的时候将这个配置文件包含到配置中去, 然后在这个system
下面任意地方用 actorOf(FromConfig.props(Props[YourClass]))
来
得到router actor. 发送任务时将任务发给router actor,它会自动转发
给cluster中的注册在 “user” 下面的注册名为 “statsWorker” 的actor
(不一定是StatsWorker,你可以随意将不同类型的Actor注册在这个名字下
面). 更多的配置,查看routing.
下面的代码是用编程的方式实现配置文件里的配置.
val workerRouter = context.actorOf( ClusterRouterGroup(ConsistentHashingGroup(Nil), ClusterRouterGroupSettings( totalInstances = 100, routeesPaths = List("/user/statsWorker"), allowLocalRoutees = true, useRole = Some("compute"))).props(), name = "workerRouter2")
Group example: Client side
object StatsSampleClient { def main(args: Array[String]): Unit = { // note that client is not a compute node, role not defined val system = ActorSystem("ClusterSystem") system.actorOf(Props(classOf[StatsSampleClient], "/user/statsService"), "client") } } class StatsSampleClient(servicePath: String) extends Actor { // servicePath is "/user/statsService", which is the second argument of the Props. val cluster = Cluster(context.system) val servicePathElements = servicePath match { case RelativeActorPath(elements) => { // println("servicePath is: "+servicePath) ; elements} case _ => throw new IllegalArgumentException( "servicePath [%s] is not a valid relative actor path" format servicePath) } import context.dispatcher val tickTask = context.system.scheduler.schedule(2.seconds, 10.seconds, self, "tick") var nodes = Set.empty[Address] override def preStart(): Unit = { cluster.subscribe(self, classOf[MemberEvent], classOf[ReachabilityEvent]) } override def postStop(): Unit = { cluster.unsubscribe(self) tickTask.cancel() } def receive = { case "tick" if nodes.nonEmpty => // just pick any one val address = nodes.toIndexedSeq(ThreadLocalRandom.current.nextInt(nodes.size)) val service = context.actorSelection(RootActorPath(address) / servicePathElements) service ! StatsJob("this is the text that will be analyzed") case result: StatsResult => println(result) case failed: JobFailed => println(failed) case state: CurrentClusterState => nodes = state.members.collect { case m if m.hasRole("compute") && m.status == MemberStatus.Up => m.address } case MemberUp(m) if m.hasRole("compute") => nodes += m.address case other: MemberEvent => nodes -= other.member.address case UnreachableMember(m) => nodes -= m.address case ReachableMember(m) if m.hasRole("compute") => nodes += m.address } }
而在客户端代码中,首先向cluster订阅成员事件来获取所有的成员. 然后
根据成员的role来筛选出相应的node. 真正开始发送任务的时候,随机排
列这些node,然后使用
context.actorSelection(RootActorPath(address) /
servicePathElements)
这句话返回一个 ActorSelection
的对象,对它
发送信息能自动地发送到某个注册在cluster上的某个node上的某个actor
上面去.准确地说,后半段确定Actor的URN,前半段应该是URL.
利用ClusterSingletonManager作为StatsService的外包,虽然每个node上
都运行了创建代码,但实际上只有一个StatsService被创建,从而实现单例.
在node被创建之后,node沟通其它node并加入cluster,随后连接上singleton
下的statsService,如果是第一个node则会创建router,然后创建3个
worker. 之后的node直接在连上statsService之后创建3个worker. worer是
由workerRouter创建的,3个worker的设定在配置文件里
max-nr-of-instances-per-node = 3
. 这个设定只对pool有效,group里
面就算设定了也不会创建worker.
singlton意味着Actor在整个集群上只有一个,目前看来是先创建的node 上面载有这个单例,如果这个node挂了怎么办?
singletonManager 被映射到 /user/singleton 上作为 StatsService 的
容器, StatsService 自然就映射到其下
/user/singleton/statsService
.
AdaptiveLoadBalanceing Pool 和 AdaptiveLoadBalancing Group 两种 实现方式.中心是cluster metrics和配置adaptive的router.
preStart
里面发送信息,这个时候路由还没有设置好,发
送的message全部/部分都会变成 dead letter.
sender()
经常会返回
deadLetter
, 因为 sender()
真正被调用的时候不是接收到信息的
时候.
Generally, we use maven to manage the directory layout. Hence, you will get a standard directory layout like below:
project src main java [scala] resources environment.properties environment.test.properties environment.prod.properties config application.yml application-dev.yml application-[profile].yml [other] test java [scala] target
The files environment.[profile].properties
are used to provide
different profiles to maven.
Since maven is used to manage the package dependencies, different maven profiles enables you to use different dependencies for development and production, e.g. run different tasks. I am not farmiliar with maven, and cannot tell how and when it can be used.
You have to add spring-boot-maven-plugin
as one of the plugins
under build
node in pom.xml.
maven clean insatll
Maven can help running test, web application and produceing reports by plugins. TODO
mvn spring-boot:run
to run the
application with default profile; mvn -Pprod spring-boot:run
to run the application with prod profile.
mvn clean package
to build the jar file
and run it through command line.
<packaging>war</packaging>
in pom.xml
file, under the root node. Then run mvn package
as
building the jar file.
From command line, you can run the jar file with specification of profile.
java -jar -Dspring.profiles.active=production YOUR_JAR.jar
The files application-[profile].yml
are used to provide different
configurations for spring.
Spring configuration files let you to inject different configurations for development and production. For example, you can use different database for development and production (different host, port, username and password, etc.).
You can get access to these configurations through many ways
@Value("${NAME:DEFAULT_VALUE}")
.
Environment
object.
// This line is not about get the property, but about define // property and let it accessible by somewhere else. @Configuration public class AppConfig { @Inject Environment env; @Bean public MyBean myBean() { MyBean myBean = new MyBean(); myBean.setName(env.getProperty("bean.name")); return myBean; } }
EnvironmentAware
interface to get
an environment object to get the property.
public class MailConfiguration implements EnvironmentAware { public void setEnvironment(Environment environment) { this.propertyResolver = new RelaxedPropertyResolver(environment, ENV_SPRING_MAIL); } }
One difference between Environment
and @Value
is that we can
access the information related to profiles. The environment can
determine which profiles are currently active, and which profile
should be active by default.
env.acceptsProfiles(String... profiles) env.getActiveProfiles() // return an array of strings
Environment
interface extends PropertyResolver
, which is used
to acess the properties. You can get property directly from the
environment object. The example above provide a way that get
properties by key name following a relaxed rule.
Different property sources are loaded in predefined order. You can set the default additional property file by
SimpleCommandLinePropertySource source = new SimpleCommandLinePropertySource(args); if (!source.containsProperty("spring.profiles.active")){ app.setAdditionalProfiles([profile_name]) }
A class is recognized as a source of configuration if it has the
@Configuration
class. @Bean
defined in this class can be
accessed from AnnotationConfigApplicationContext
. It is a
replacement of XML configuration files that defines beans.
Get the configuration from configuration class in some where
else.
public class ConfigurationTest { @Test public void testHelloworld () { AnnotationConfigApplicationContext ctx = new AnnotationConfigApplicationContext(AppConfig.class); Assert.assertEquals("hello", ctx.getBean("message")); // message is the name of the Bean method in configuration class // it can also be used to get any kind of object } }
AppConfig
is the class that used as configuration source.
In configuration source, add
@EnableConfigurationProperties(A.class)
. In target class A
,
add @ConfigurationProperties(prefix = "PREFIX")
. Then the
properties under PREFIX
will be automatically assigned to the
attributes of class A
. DETAILS OF THE CONFIGURATION SOURCE???
@EnableAutoConfiguration
auto configure the application. I
don’t know the details of this annotation, but it auto wires the
configurations into the embedded tomcat. Besides, it also
configures the database (host and port) automatically when
database template is injected in the project.
It is mainly implemented by HttpSecurity
, which can be injected
and configured in method of class that extends
WebSecurityConfigurerAdapter
.
public class SecurityConfig extends WebSecurityConfigurerAdapter { @Inject private UserDetailsService userDetailsService; // class access to the user database @Value("${drill.security.rememberme.key}") private String remerberMeKey; // key for cookies @Inject public void configureGlobal(AuthenticationManagerBuilder auth) throws Exception { auth .userDetailsService(userDetailsService) // the point that connect the authentication module and user database .passwordEncoder(passwordEncoder()); } @Override public void configure(WebSecurity web) throws Exception { // Spring security ignore following URL web.ignoring() .antMatchers("/bower_components/**") .antMatchers("/fonts/**") .antMatchers("/images/**") ...; } @Override protected void configure(HttpSecurity http) throws Exception { http.authorizeRequests() .antMatchers("/app/**").authenticated() .antMatchers("/app/rest/register").permitAll() .antMatchers("/app/rest/logs/**").hasAuthority(Authority.ADMIN) // set authorities .and() .formLogin() .usernameParameter("username") .passwordParameter("password") .loginProcessingUrl("/app/authentication") .successHandler(ajaxAuthenticationSuccessHandler) .failureHandler(ajaxAuthenticationFailureHandler) .permitAll() // set login service url .and() .logout() .logoutUrl("/app/logout") .logoutSuccessHandler(ajaxLogoutSuccessHandler) .deleteCookies("JSESSIONID") .permitAll() .and() .exceptionHandling() .authenticationEntryPoint(authenticationEntryPoint) .and() .rememberMe() .rememberMeServices(rememberMeServices) // it is not persistent token .key(remerberMeKey) .and() .csrf().disable() // not clear .headers().cacheControl(); // not clear } }
The foundation of user securing is about session and cookies. Once the user logged in, the server will set the cookies in browser. Then browser will send the cookies via HTTP protocol everytime it send new request to the server. On the server side, session is also kept with a unique random id to identify the client. Look another blog Init the project to see how to replace the in memory user store by a repository that connects to database. On the server side, programmer can get the user information by:
public static String getCurrentLogin() { SecurityContext securityContext = SecurityContextHolder.getContext(); Authentication authentication = securityContext.getAuthentication(); UserDetails springSecurityUser = null; String userName = null; if(authentication != null) { if (authentication.getPrincipal() instanceof UserDetails) { springSecurityUser = (UserDetails) authentication.getPrincipal(); userName = springSecurityUser.getUsername(); } else if (authentication.getPrincipal() instanceof String) { userName = (String) authentication.getPrincipal(); } } return userName; }
This example uses WebMvc framework, which uses
WebMvcConfigurerAdapter
and ViewControllerRegistry
to connect
the urls and html templates.
Add spring-boot-starter-security
into dependency, make
WebSecurityConfig.java
.
@Configuration @EnableWebMvcSecurity public class WebSecurityConfig extends WebSecurityConfigurerAdapter { @Override protected void configure(HttpSecurity http) throws Exception { http .authorizeRequests() .antMatchers("/", "/home").permitAll() .anyRequest().authenticated() .and() .formLogin() .loginPage("/login") .permitAll() .and() .logout() .permitAll(); } @Autowired public void configureGlobal(AuthenticationManagerBuilder auth) throws Exception { auth .inMemoryAuthentication() .withUser("user").password("password").roles("USER"); // user repository in memory with username = "user" and password = "password" // it is just for demon } }
Maven projects are defined with an XML file named pom.xml. This file gives the project’s name, version, and dependencies that it has on external libraries. Here is an example.
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>org.springframework</groupId> <artifactId>gs-maven</artifactId> <packaging>jar</packaging> <version>0.1.0</version> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <version>2.1</version> <executions> <execution> <phase>package</phase> <goals> <goal>shade</goal> </goals> <configuration> <transformers> <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"> <mainClass>hello.HelloWorld</mainClass> </transformer> </transformers> </configuration> </execution> </executions> </plugin> </plugins> </build> </project>
<modelVerion>
POM model version
<groupId>
Domain name of the group or organization.
<artifactId>
Name to the project’s library artifact (name of
JAR file)
<version>
Version of the project
<packaging>
How it should be packaged (in JAR or WAR file)
mvn compile mvn package mvn install
The first command compiles the project, and the .classfiles shold be
in the target/classes directory.
The second command compile the code, run tests and finish by
packaging the code up in a JAR file in the target directory.
Tht third command does the same thing as the second command, then
copy the JAR file into the local dependency repository, under the
directories with name of groupId, artifactId and version. So on my
machine, its location is
~/.m2/repository/org/springframework/gs-maven/0.1.0/gs-maven-0.1.0.jar
.
Add dependencies of project into the pom.xml file, in the
<project>
element.
<dependencies> <dependency> <groupId>joda-time</groupId> <artifactId>joda-time</artifactId> <version>2.2</version> </dependency> </dependencies>
This description will tell the maven to get the joda-time package
as external library. You can specify a <scope>
element to specify
if the dependencies are required for compiling the code but will be
provided during runtime by provided
; or decalre the dependencies
are only necessary for compiling and running tests.
When I compile the command, maven downloads the joda-time pakcage from https://repo.maven.apache.org/maven2/joda-time/. Does the maven use its domain when the group id does not contain the domain by default?
You can also create a project from scratch
mvn archetype:generate -DgroupId=com.mycompany.app -DartifactId=my-app -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false
mvn scala:console
can launch the scala console with the pom
configurations (you have to specify maven-scala-plugin).
project src main java [scala] resources environment.properties environment.test.properties environment.prod.properties config application.yml application-dev.yml application-[profile].yml [other] test java [scala] target classes the classes with the same structure as src/main the directories in src/main/resources test-classes the classes with the same structure as src/mainproject xxx.jar target
Honestly, I don’t understand it. But I collect my thoughts about it here. First, the monda is used to chain operations and if the first (preceding) operation fails, the whole chain stops. An example that I understand is the usage of for-expression with pattern match. If the pattern matches, go next operation, else skip this element.
Another point of view is about the unit operation. A monad flatMap with its unit operation will return the monad itself.
My conclusion:
A monad is a container that supports operation chain and follows
the monad laws. The basic operations of this container are
flatMap
and unit
, where flatMap
actually accepts a function
that maps each element to a container, then binds these containers
together; unit
returns a container given an element. Monad laws
guarantee the reliability of operation chain on these contains.
Monad seems like a design pattern that enables the operation chain
along with pattern match in functional programming. Because
flatMap
chain with different functions is equivalent to nested
flatMap
, the for-expression is a syntax sugar based on flatMap
chain. The advantage of this design pattern is avoding side effect.
Generator is created by yield
as well as
for-expression. Pay attention, there is no generator
class/type/trait. The genrator created through for-expression has
the same type of the sequence/iterator/stream used in
for-expression. That proves the for-expression is a syntax sugar of
monad types. So, guess what will happen when we create a
for-expression using both list and stream?
val fibo:Stream[Int] = 1 #:: 1 #:: fibo.zip(fibo.tail).map(x=>x._1+x._2) val l = List(1,2,3,4,5) val lsg = for{x <- l; fib <- fibo} yield (x,fib)// infinite loop, crash the software/your machine val slg = for{fib <- fibo; x <- l} yield (x,fib)// return a stream
Following the definition of Monad, the first mixture lsg
actually
makes l flatMap (x => fibo)
, which returns a List that tries to
expand infinite stream fibo
, hence block your machine. The second
mixture slg
returns a Stream that expand the list l
, hence,
works well. Besides, I have to clarify one thing: different monads
demonstrated above all accept GenTraversableOnce
as parameter and
return monad of their own type. That is why they can be mixed
together and the first expression decides the type of the final
output of for-expression.
In the example provided by the oneline course, they used JUnit,
ScalaTest and ScalaCheck together. In their example, the class
of Properties
is in src/main/scala
and is called by another
class defined under src/test/scala
. In the class under test
folder, many instances of the Properties
class are created to
check different children classes of the target class.
abstract class QuickCheckHeap extends Properties("Heap") with IntHeap { property("min1") = forAll { a: Int => val h = insert(a, empty) findMin(h) == a } lazy val genHeap: Gen[H] = for { a <- arbitrary[Int] h <- oneOf(value(empty), genHeap) } yield insert(a, h) implicit lazy val arbHeap: Arbitrary[H] = Arbitrary(genHeap) }
@RunWith(classOf[JUnitRunner]) class QuickCheckSuite extends FunSuite with Checkers { def checkBogus(p: Prop) { var ok = false try { check(p) } catch { case e: TestFailedException => ok = true } assert(ok, "A bogus heap should NOT satisfy all properties. Try to find the bug!") } test("Binomial heap satisfies properties.") { check(new QuickCheckHeap with BinomialHeap) } test("Bogus (1) binomial heap does not satisfy properties.") { checkBogus(new QuickCheckHeap with Bogus1BinomialHeap) } test("Bogus (2) binomial heap does not satisfy properties.") { checkBogus(new QuickCheckHeap with Bogus2BinomialHeap) } test("Bogus (3) binomial heap does not satisfy properties.") { checkBogus(new QuickCheckHeap with Bogus3BinomialHeap) } test("Bogus (4) binomial heap does not satisfy properties.") { checkBogus(new QuickCheckHeap with Bogus4BinomialHeap) } test("Bogus (5) binomial heap does not satisfy properties.") { checkBogus(new QuickCheckHeap with Bogus5BinomialHeap) } }
It is a tool for property-based testing for scala. It has NO
external dependencies and integrated in the test frameworks
ScalaTest. It can also be used standalone, with its built-in test
runner.
First, create a class that extends class
org.scalacheck.Properties
with the name of data object that you
want to test. It is used for the library to generate test data to
test your algorithm.
Second, create test case by
property("NAME_OF_FUNCTION") = forAll{
CONTAINER_OF_DATA => TEST_CASE_WITH_TARGET_FUNCTION
}
forAll
is the function org.scalacheck.Prop.forAll
.
Third, to run the tests, you can put the properties in
src/test/scala
then use test task to check them.
This is a test framework that integrates all together. ScalaTest
provides many test styles: FunSuit
, FlatSpec
, FuncSpec
,
WordSpec
, FreeSpec
, Spec
, PropSpec
, FeatureSpec
. They
are mainly different on syntax styles. It is recommanded that the
user creates a base class that extends all the required features
(as a subset of all the provided features) and extends this base
class through everywhere of the project to make the style
consistent.
This class uses the junit framework for unittest. JUnit will
invoke the class with @RunWith
annotation (extend by type
hierarchies) to run the tests. The annotation parameter in the
example is a class of the type JUnitRunner
. In fact, any class
extends Runner
is acceptabel. Notice that function classOf
acts the same as obj.getClass()
.
Maven Running tests with command mvn test
. In fact, you
cannot run the tests with the example above. Solution is to
change the name QuickCheckSuite
to QuickCheckSuiteTest
or
TestQuickCheckSuite
or QuickCheckSuiteTestCase
to run the
tests. Even I did not explicitly specify plugin surefire
, maven
uses this plugin to run test with my configuration. By default,
maven following name convertions when looking for tests to
run. So I have to change the name or modify the surefire
configuration to apply new name convertion rules.
The annotation uses class of JUnitRunner
as parameter. This
class is provided by scala-test framework that connect
scala-test and junit.
To work with ScalaCheck, you have to mix the trait
org.scalatest.prop.Checkers
to your test class. Checkers
class contains check
method to perform ScalaCheck property
checks, which are provided in the class QuickCheckHeap
in the
example above. In fact, you can also use the check
method in
JUnit directly. This method will generate a large number of
possible inputs to look for an input that the property does not
hold.
Exception is handled by Try[T], which is used as return type like Option. But Try[T] maches Success with result or Failure with throwable. As Try[T] is also a monad, operations of element T are usually propagated to monad methods of Try[T] to get a new Try[U], map Try[T] by f should return a new Try[f(T)].
Future[T] is a container of Try[T]. User provide operations/callbacks that deal with Try[T] and future will call them at certain time. Future and Try form a nested monad, monad operations are propagated and you cannot see direct operation on T in the source code of scala library. But take it easy, usually, you won’t have to do it yourself.
To get/compose a future (that can be our job), therer are four methods:
Future
such as flatMap
flatMap
, binary bind operation of futures. Register promise
to this when failed of exception of call funtion; register
promise to that when successed.
onComplete
, parameter is a function Try[T] => U
. This is
a funtamental funciton of map
, foreach
, failed
, etc. It
propagate the Try mechanism.
collect
recover
fallbackTo
zip
andThen
Promise
acts like a mailbox, to compose a new future, the old
future has to put the result, which is a Try[T], into the
promise by complete
method. Then the new future will call its
callback when the complete
of promise is called.
In my opinion, promise can be merged with feature as one
unit. Because it is more straight to think in the way that:
feature calls the callbacks when it is completed. In fact,
the DefaultPromise
is actually extends the Future
trait. Of
course, the designers of scala have their proper reason.
Currently, I think promise is used in race (concurrent processing of futures results) or to provide a new future for initailization of fold operations.
for-expression
, which is a syntax sugar of flatMap
async...await
grammar.
Try
and Future
are dual, so as Iterable
and
Observable
. For iterable, you can get an iterator and using
hasNext
and next
to get elements. For Obserable
, you use
Subscribe
and onNext
, onComplete
to get elements.
flatten
after map
returns an observable that merges several
streams randomly. concat
after map
returns an observable, in
which, elements of each stream are grouped together sequentially
as well as the order of the streams.
Don’t understand well the way of creating observable with a function from Observer to Subscription -_-!
It works like promise, which is used as bridge or chain between Obsever and Observable. Four types of subjects are provided:
Like Try
a handle that enables process to stop receiving messages from Observable.
See example POM Configuration.
Fully qualified name of project is “<groupId>:<artifactId>:<version>”. If groupId is a domain, it will be transformed into directory path while downloading. Download something with groupId=org.X.A from url.com will download things from url.com/org/X/A in the end.
When we have two projects, A and B, A is supposed to be parent
project of B. You have to specify the pom.xml of project B to
indicate that the parent of B is A with the groupId,
artifactId and version of A. You can remove the groupId and
version of B to make A and B have the same /groupId and
version. One can also indicate the <relativePath>
for parent
project to put the A and B in the same directory level.
B inherite the dependencies from A.所以parent用来继承POM,dependency 用来解决代码依赖. parent的项目在代码上没有关系.
The packages downloaded by maven are put under HOME.m2/repository/.
所有 POM.xml 里面的元素都可以用 ${project.A.B.C}
的方式进行引用.比
方说如果在 XML 根路径下有这样的结构
<A><B>lalala<C>blabla<C/></B></A>
, ${project.A.B.C}
就等于
blabla
.
首先,maven可以使用一系列的plugin来支持各种功能,包括发布,运行测试等
等.在使用 spring-boot-maven-plugin
的时候,可以用 mvn
spring-boot:run
来运行程序. spring-boot
应该是plugin的名字, run
则是定义好的task. 在使用 maven-scala-plugin
的时候,如果加入以下设
置
<executions> <execution> <goals> <goal>compile</goal> <goal>testCompile</goal> </goals> </execution> </executions>
运行 mvn scala:testCompile
则只会编译test部分的代码.
val f = (x:Int) => x*x
One difference between def f = ...
and val f = ...
is: def
is evaluated on call and returns a new function instance every
time.
val test: () => Int = { println("val") val r = util.Random.nextInt () => r } // returns the same result every time you call test(), // "val" will be printed after definition def test: () => Int = { println("def") val r = util.Random.nextInt () => r } // returns the difference results every time you call test(), // "def" will be printed every time you call it. val tt = test // acts like the first example val test = () => util.Random.nextInt def test = () => util.Random.nextInt // they are the same from behavior, you have to call the function by test() // the def declaration actually defined a function that returns a function. // if you type test without parenthesis, you will receive a function value. def test = util.Random.nextInt // this line directly assign the test function to the nextInt function, // you can call the function directly by name without the parenthesis
method/expression starts by def
will be evaluated when you call
obj.f
while those starts by val
will be evaluated
once. In addition, if you declare a val
that assigns to def
,
the def
will be evaluated immediately, and the new val
acts
like the first case in the examples above.
val l = List(1,2,3) l.map((x) => x*x) l.map( x => x*x) // ignore the parenthese l.map( _ % 2 == 0) // ignore the parameter, using the placeholder for expression l.map(f(_)) // using the placeholder for function l.map(f) // ignore the placeholder for function
abstract class is like Java, trait is like interface in Java. However, trait can have parameters and defined methods. In addition, trait is not abstract class since it cannot has constructors.
It is not true for mutable collection.
var s = new Array[Man](new Man()) var t:Array[Human] = s t[0] = new Woman() var m:Man = s[0]// what is it? man or woman?
But we can use imutable collection.
The principle is: child type can do anything that can be done by parent type.
To avoid conflict, the compile force the +T can only be used for returned type, the -T can only be used for argument type. A method to by pass this problem is using boundary.
class List<+T>{ def prepend[U >: T](U elem) }
Notice that in generic type definition, one can use A>:B
and
A<:B
to add constraint to the meta type.
Case class enables you to make classes with different parameters
abstract class CodeTree case class Fork(left: CodeTree, right: CodeTree, chars: List[Char], weight: Int) extends CodeTree case class Leaf(char: Char, weight: Int) extends CodeTree
For the classes listed above, you cannot directly access to any
paramter (field) of Fork
or Leaf
. You have to use case
Fork(a,b,_,_)
to access to the paramters.
case
match can also be used in lambda function as a short
version.
List.map(p => p._1 * p._2) List.map({case (x,y) => x * y}) //List.map{case (x,y) => x * y} also works
Option calsss used for the condition that the returned type can be empty/null, etc.
If you want to match a variable in context, you have to weither use
a capitalized variable name Variable
or wrap it by backticks
`variable`
.
log_{32}(N)
. Important functions: map
, filter
,
reduceRight
, reduceLeft
, foldRight
, foldLeft
, flatten
,
flatMap
, take
, drop
, takeWhile, =dropWhile
, span
,
group
, groupBy
.
Sequence as parameter list is represented by this(par: T*)
.
+
and ++
operations overwrite the existing key.
Map(k1->v1, k2->v2, k3->v3)
withDefaultValue
, this function returns
a new map (everything in this course are immutable).
lazy
Stream, acts like sequence, but it does not evaluate until be called. It can be used to ignore unecessary computation and coding infinite concepts.
val fibo:Stream[Int] = 1 #:: 1 #:: fibo.zip(fibo.tail).map(x=>x._1+x._2) println(fibo.take(10).toList) def from(n: Int): Stream[Int] = n #:: from(n+1) // infinite integer stream starts from n // recursive stream def sieve(s: Stream[Int]): Stream[Int] = s.head #:: sieve(s.tail filter (_ % s.head !=0)) // stream of primes 质数 val primes = sieve(from(2)) primes.take(10).toList // the first 10 primes starts from 2.
Three ways of using stream
elem #:: Stream
Because Stream has consistent API as List/Seq/Vector, you can use it as if you have a collection that contains everything.
The difference between stream and iterator is stream memories the values while iterator doesn’t.
Iterator can be used in for-expression. For-expression can also use pattern match.
for{ pattern(x) <- seq; pattern2(y) = x(k); ...} yield ...
As show in the example above, you can apply pattern to loop on the elements of sequence, you can even add some simple statements in the for-expression. It is equivalent to:
seq.withFilter({case pattern(x) => true; case _ => false}) .map({case pattern(x) => x(k)}) .withFilter({case pattern2(y) => true; case _ => false}) .map({case pattern2(y) => y}) ...
The function withFilter
returns a FilterMonadic
, which
provides four functions flatMap
, map
, foreach
,
withFilter
. It works like list of call backs in programming
point of view.
From wikipedia:
/In functional programming, a monad is a structure that represents computations defined as sequences of steps: a type with a monad structure defines what it means to chain operations, or nest functions of that type together.
lazy val
lazy val
is only evaluated once when called. You can also define
a lazy parameter by def myFUnc[T](param: => T)
, then the
parameter will be evaluated when it is called in myFunc
if the
param is returned by an expression/function.
Two-phase commits is used to update multiple documents as a fake atomic operation. The principle of two-phase commits is creating temporary, inter-media records to support rollback operations
The first method is adding a label to indicate the current accessing application.
The second method is using the old value of files to update as a part of query to ensure that the target fields of current document is not updated.
var myDoc = db.COLLECTION.findOne(condition); if(myDoc){ var oldValue = myDoc.value; var results = db.COLLECTION.update( { _id: myDoc._id, value: oldValue }, { $inc:{value: -fee} } ) }
Another possible option is: if you want to deduct an amount of money from an account and the rest money should not be negative, you can use following commands:
db.COLLECTION.update({_id: id, value: {$gte: fee}}, {$inc:{value: -fee}})
It means the account will be updated only if the value is enough.
The third method is add an unique field (version) to the document.
Example: patron and address. User informations should be put together, such as the account info (password, email) and application relative info (balance in account).
It is used for one-to-few such as the authors of a book. One can use array in a document to model such relations. However, if the values of embedded elements are few, it is better to save them in another documents using references. For example, the publishers of books. Another principle is DO NOT create large arrays in document. Large arrays in document is inefficient for three reasons:
Save each node as a document that contains the reference to the parent
Save each node as document that contains an array of references to children nodes.
It is also useful to add additional relations such as ancestors of nodes into the document.
Another way to accelerate the searching of a tree is
materialize paths. This method add a path
attribute that
describe the path using a string. Searching such string requires
using of regular expression, but is faster than the previous
solution.
This method is tricky. It likes a binary tree that use values to indicate the relative nodes.
db.categories.insert( { _id: "Books", parent: 0, left: 1, right: 12 } ) db.categories.insert( { _id: "Programming", parent: "Books", left: 2, right: 11 } ) db.categories.insert( { _id: "Languages", parent: "Programming", left: 3, right: 4 } ) db.categories.insert( { _id: "Databases", parent: "Programming", left: 5, right: 10 } ) db.categories.insert( { _id: "MongoDB", parent: "Databases", left: 6, right: 7 } ) db.categories.insert( { _id: "dbm", parent: "Databases", left: 8, right: 9}) var databaseCategory = db.categories.findOne( { _id: "Databases" } ); db.categories.find( { left: { $gt: databaseCategory.left }, right: { $lt: databaseCategory.right } })
One can retrieve the children nodes by using the left and right indicators.
One of the main benefit of indexing is sorting, the application can quickly find the matched document by traveling through the ordered indices very quickly
db.COLLECTION.ensureIndex({key1: 1, key2: -1})
It can help the sort
funciton using {key1: 1, key2: -1}
,
{key1: -1, key2: 1}
, but cannot help sorting with the key
{key1: 1, key2: 1}
.
Index also support properties like:
update
, findAddUpdate
, remove
are atomic. Put fields in one
document can ensure the write operations are atomic.
Keyword search cannot provide NLP/IE functions, such as stemming, synonyms, ranking, which are usually required by search engine.
$
, cannot contain .
, cannot
contain null
.
_id
field may be any BSON data type except the array. It is
ObjectId
by default.
DBRefs are used to representing a document rather a specific reference type. It contains the name of collection (optional: database name).
db.COLLECTION.insert(doc/docs)
You can insert one document or a list of documents. When you insert a list of documents, the operation is not atomic. The returned result shows the statistics of the write operations including the write errors and the number of successfully inserted documents.
db.COLLECTION.find({name: value, 'name1.name2.name3':value/condition})
Check all the elements in array when ‘name#’ corresponds to array. Special conditions
$in, $ne, $nin, $lt, $gt, $lte, $gte
$and, $or, $not, $nor
$exists, $type
$regex, $text, $where, $mod
$all, $elemMatch, $size
$, $slice
Difference between with/without $elemMatch
is that: $elemMatch
restrict that one element in array should match all the conditions,
while without $elemMatch
, different elements in the array can
match different part of the conditions.
Match element / array can be done by query {name: value}
when value
is a single variable/object or an array.
Elements in array can be retrieved by index
db.COLLECTION.find({'name.0': value})
.
db.COLLECTION.find(***, {name:1})
db.COLLECTION.find(***, {'name1.name2':1})
This projection will returns all the elments with the path ‘name1.name2’ when any one in the path represents an array.
db.COLLECTION.find(***, {name1: {$elemMatch: {name2:value}}})
$elemMatch
can be used in the second argument to limit the
returned elements in array. However, it CANNOT be used in nested
array and only returns the FIRST element.
Aggregation framework/unwind. Aggregations are operations that process data records and return computed results. Pipeline
You can expand the nested arrays by several $unwind
operations to get a large number of documents that each one
contains a single combination of elements in different levels
of nested arrays. Then match the elements you need and maybe
regroup them together. However, if you want to aggregate the
matched elements in nested arrays into array after $unwind
operation, it is very complex.
However, you can just return the tasks by $group
operation
that matches, for example, the nested element attributes.
db.COLLECTION.update(find condition, {$set: {name: value}})
$inc
increase number, etc. upsert
will insert element when it
does not exist in collection.
db.COLLECTION.update(find condition, {$push/$pull: value})
db.students.update( { _id: 1 }, { $push: { scores: { $each: [ { attempt: 3, score: 7 }, { attempt: 4, score: 4 } ], $sort: { score: 1 }, $slice: -3 } } })
The command above append 2 elements, then sort them by ascending score, then keep the last 3 elements of the ordered array.
db.COLLECTION.update({"name1.name2":value}, {$set:{'name1.$.name2:value}})
It only works for one match.
If you want to update multiple elments, use forEach function.
db.Projects.find().forEach( function(pj){ pj.groups.forEach( function(gp){ gp.tasks.forEach( function(task){ if (task.name.match(/update/)){ task.weight=5; } }); }); db.Projects.save(pj); } )
Impossible for only one query. The issue is reported on 2010, but is still there on 2015…
Functional key words:
$currentDate, $inc, $max, $min, $mul, $rename, $setOnInsert, $set, $unset
$, $addToSet, $pop, $pullAll, $pull, $pushAll, $push
$each, $position, $slice, $sort
db.COLLECTION.remove(condition)
remove all documents when condition is not provided.
db.COLLECTION.drop()
use DATABASE
db.dropDatabase()
db.COLLECTION.update(find, {$unset: {name: value}})
value can be 1, “”, true/false.
db.COLLECTION.update({"name1.name2":value}, {$unset:{'name1.$.name2':1}})
show dbs
use DB
show collection
help
db.help()
db.COLLECTION.help()
count()
// count of documents returned in a curcor
explain()
// report
hint()
// Forces MongoDB to use a specific index for a query
limit()
// constraints size of returned results
skip()
// skip the first number of documents
sort()
toArray()
db.COLLECTION.mapReduce(mapFunction, reduceFunction, { query: "query performed at the beginning", out: "out collection name", finalize: finalizeFunction })
Query -> map -> reduce -> finalize -> out.
在工具描述中,主要介绍经常用到的库,框架,前后端开发用到的平台.
Node.js 是一个JavaScript的平台,它提供了一个脱离浏览器的JavaScript运 行环境,同时也提供了一系列运行库和各种包. npm(nodejs package manager)则是这各平台的包管理系统.它通过读取package.json文件来安装项 目所需的依赖(就跟maven的pom.xml一样).
bower ,是nodejs之中的一个组件,本身用来专门管理前端开发所需要的包 (JavaScript,CSS).它与npm的区别在于管理对象的不同以及bower使用的是扁 平化的依赖关系而npm使用树形依赖.bower读取bower.json来安装依赖.
yoeman , 安装时叫做 yo,是一个用来创建项目骨架的工具.它根据 yeoman-generator,也就是模板来创建文件架结构.模板需要用npm来安装,比 方说
npm install -g generator-gulp-angular
模板名字以 generator
开头.
使用时运行
mkdir [app-name] && cd $_ yo gulp-angular [app-name]
来生成一个以generator-gulp-angular为模板的项目骨架,项目名字为
[app-name]
. 就刚刚使用的这个模板为例,它在创建的时候会询问你需要什
么样的angular版本,哪一种CSS样式库(bootstrap, material),哪种angular
的实现方式(angular-strap),等等.
grunt / gulp 用来运行打包,压缩,发布等任务.在刚刚的例子中,我们使 用的是基于gulp的模板,所以要用gulp来运行以上任务.生成模板的事后,已经 在项目目录下建立了一个名为gulp的目录以及一个叫做gulpfile.js的文 件.gulp目录下面包含一系列js文件. gulpfile.js里面引入gulp库,定义一些 目录(src)的路径,并引入gulp目录,从而使得gulp目录下的js文件被gulp用来 进行相应的操作.这个模板的github说明上还指出一系列可选的特性,但是目 前看不出来在哪里定义的.
.jade
.haml
HTML本身是静态的,对于现在的动态网页无能为力.为了弥合静态和动态之间
的鸿沟,有两种静态方式:库(jQuery
),框架(durandal
,
ember
). AngularJS
则提供了第三种方式,通过数据绑定等方式让浏览
器学会新的语法. AngularJS
是完全基于前端的,适用于CRUD应用.
AngularJS中的模组类似于名字空间,在几乎所有东西都在名字空间下注册.这 些注册的东西是通过名字推断来进行绑定(引用)的.AngularJS自带的那功能,都 是可以直接引用的.自己定义的东西,也可以直接靠名字推定,但是要给模组 加上依赖关系.
service, directive, filter 都是通过 factory/directive/filter
方
法注册在模组中的,这些方法统称工厂(factory)方法.
我们可以通过 config/run
方法注册在模组配置和运行的时候需要运行的
函数,这些函数也同样的可以被以类似上面的方式来调用.这些方法被归类为
模组(module)方法.
控制器(controller),被解释为classes或者constructor function,同样需 要注册到模组中,通过名字来引用.
这些东西在引用的时候,可以直接当函数参数名,可以用 $inject 来显示 说明,也可以用数组.后两种基本上是在第一种的基础上多写些代码来说明.直 接用函数参数名是最简单的,但是不被官方所推荐.
传统模板是通过模板Tempalte和模型Model来建立视图View. AngularJS是通过模
板建立起View和Model的对应,让View跟Model直接互动.
AngularJS中,template是包含AngularJS元素和属性的HTML.template中可用
的AngularJS元素和属性为 Directive, Markup (), Filter,
Form controls.通过这些template,JS(model)和(view)被连接了起来(数
据绑定).
上下文(Scope)提供表达式的运行环境,它被组织成一个类似DOM的层级结构.上 下文监视表达式并传递事件.API($watch)监视模型的变更,($apply)将任意 由Angular之外的事件对模型的改变传递到View.这里所谓的Angular之外的 事件,指的是controllers,services,event handlers.
上下文之间可以嵌套,嵌套的上下文有两种 child scope 和 isolate scope, child scope 继承上层上下文的内容, isolate scope 不继承. 如果上下文中没有对应的属性,表达式是没有任何作用的.
上下文可以看做将应用控制器controller和视图view联系起来的胶水.
在template连接的时候,directive会设置上下文中的$watch,$watch会通知
directive任何属性的变更,从而使directive能根据更新的数据来渲染DOM
控制器controller和directive都能直接访问scope,但是不能直接访问对方.这
种设计孤立controller和directive. $rootScope
提供对root上下文的访
问,一般指向ng-app所在的范围.
如果用chrome的inspect element功能查看页面元素,会发现
class="ng-scope ng-binding"
之类的属
性.Anuglar会自动将 ng-scope 加入制定了上下文的元素, ng-binding
则说明这里面的内容是用绑定实现的.
在不同层级使用同一个controller,
<div ng-controller="EventController"> Root scope <tt>MyEvent</tt> count: <ul> <li ng-repeat="i in [1]" ng-controller="EventController"> <button ng-click="$emit('MyEvent')">$emit('MyEvent')</button> <button ng-click="$broadcast('MyEvent')">$broadcast('MyEvent')</button> <br> Middle scope <tt>MyEvent</tt> count: <ul> <li ng-repeat="item in [1, 2]" ng-controller="EventController"> Leaf scope <tt>MyEvent</tt> count: </li> </ul> </li> </ul> </div>
angular.module('eventExample', []) .controller('EventController', ['$scope', function($scope) { $scope.count = 0; $scope.$on('MyEvent', function() { $scope.count++; }); }]);
这个例子中,首先我们看JS的部分.这部分注册了一个EventController,这个 controller对于给定的上下文 $scope 设定属性 count,同时注册一个 名为 MyEvent 的事件,调用一个闭包函数对于当前上下文进行操作.
接下来看HTML,里面把同一个controller用了三次,也就意味着对于
eventExample,给予三个不同的 $scope.而这三个 $scope 各自拥有自
己的 MyEvent 函数副本.最后, $emit('MyEvent')
将调用这个函数的
动作传递给上层 $scope, $broadcast('MyEvent')
则传递给下层
$scope.
$scope 的生命周期中,模型的改变不会立即触发 $watch, 而是通过在 $apply 结束时的 $digest 来触发 $watch,这么做的好处在于将多个 改变合为一个 $watch 动作. 当子上下文不需要继续存在时,子上下文的 创建者负责调用 scope.$destroy() 负责停止事件传递,允许垃圾回收来 释放内存.
angular.module会覆盖已存在的名字.模组提供 provider, factory, service, value, constant, animation, filter, controller, directive, config, run 这些方法.除了最后两个以 外,都是注册某一种功能. provider,factory,service,value,constant 都是JS内部用的, directive,filter,animation,controller 都是在html里面用 的(JS里面定义的).
Provider是AngularJS的基础,Value,Factory,Service,Constant都建立在Provider之上.下
面的 myApp.XXX
方法实际上都是调用 $provider.XXX
.
var myApp = angular.module('myApp', []); myApp.value('clientId', 'a12345654321x'); myApp.controller('DemoController', ['clientId', function DemoController(clientId) { this.clientId = clientId; }]);
<html ng-app="myApp"> <body ng-controller="DemoController as demo"> Client ID: </body> </html>
其实就是定义一个模组下的变量/对象,可以是简单值,也可以在这个变量 下加入函数.
myApp.factory
方法.返回的是一系列变量,函数.
myApp.service()
方法,注册一个构造函数,得到一个类(class),
用 new 来得到新实例.返回的是构造函数.
$get
方法,DI会把这个方法返回的东西绑
定到注册的Provider名字上.
myApp.provider('AAA', function AAAProvider(){ this.configFunc = function(bla){blabla}; this.$get = function(lab){ return BBB}; }); myApp.config(function(AAAProvider){ AAAProvider.configFunc(...); }); myApp.controller('CCC', function(AAA){ /*AAA is BBB*/});
provider 只能在config里面,而在run里面,只能用非 provider.
myApp.constant
来注册.
服务都是惰性加载和单例,通过 factory 注册.服务本身对于html是不可 见的.在JS里面引用其功能. 除了用工厂方法意外,还可以在模组的config方法中通过 $provie.factory 来 注册服务.这里的Service和前面Provider段落中的不是一个概念.
控制器是用于连接上下文和服务的,通过 controller 注册,对于html可见,通 过 ng-controller 来进行绑定.本身也可以定义一些行为.不需要返回什 么东西.
Directive提供自己定义的HTML标记,基于AngularJS HTML compiler
($compile
). ngApp, ngRepeat 等等就是AngularJS自带的
directive,这些directive通过 ng-app, ng-repeat 标记在HTML文件中
改变元素的行为模式.当然,这里的compile并不是真正意义上的编
译,AngularJS只是将事件监听器绑定到HTML元素上,使得这些元素可以互动.
在 $compile
的API中可以找到directive的各种option的详细说明.
命名方面, -,:,_
都可以作为分隔符, ng-model
和 ng:model
是等
效的.同时还可以在前面加上 x-
或者 data-
前缀, ng-model
和
data-ng:model
是等效的.
使用directive可以将它作为tag name, attribute name, comment 或者 class name.
<my-dir></my-dir> <span my-dir="exp"></span> <!-- directive:my-dir exp--> <span class="my-dir: exp"></span>
以上三种标记方式是等效的.当然,推荐使用tag name和attribute name.
ngAttr 能够绕过浏览器限制使用AngularJS的表达式设定attribute.
<circle cx=""></circle> <circle ng-attr-cx=""></circle> <svg viewBox=""></svg> <svg ng-attr-view-box=""></svg>
Directive返回固定的option.如果说Service(Factory)是注册名字和对应的属性,函数;Directive则是将值和函数绑定到预先定义好的选项
module.directive 函数注册directive,它接受一个名字和一个工厂方法.方 法返回template,用其填充directive元素的内容.
值为 ‘A’ (match attribute), ‘E’ (match element), ‘C’ (match class).指定这个directive被用作元素的属性,元素名或类
操作DOM.这个option需要指定一个函数,函数的参数为 scope (上下文),element (当前元素),attrs (当前元素的属性), ctrl (array of controllers required by the directive).
angular.module('docsTimeDirective', []) .controller('Controller', ['$scope', function($scope) { $scope.format = 'M/d/yy h:mm:ss a'; }]) .directive('myCurrentTime', ['$interval', 'dateFilter', function($interval, dateFilter) { function link(scope, element, attrs, ctrl) { var format, timeoutId; function updateTime() { element.text(dateFilter(new Date(), format)); } scope.$watch(attrs.myCurrentTime, function(value) { format = value; updateTime(); }); element.on('$destroy', function() { $interval.cancel(timeoutId); }); // start the UI update process; save the timeoutId for canceling timeoutId = $interval(function() { updateTime(); // update DOM }, 1000); } return { link: link }; }]);
设定函数 updateTime 直接操纵元素内容,通过 $watch 将元素中的属 性变动跟操作函数绑定在一起. 通过 $interval 来实现自动读秒更新时 间. 绑定 $destroy 来终止 $interval 读秒.
angular.module('docsIsolateScopeDirective', []) .controller('Controller', ['$scope', function($scope) { $scope.naomi = { name: 'Naomi', address: '1600 Amphitheatre' }; $scope.igor = { name: 'Igor', address: '123 Somewhere' }; }]) .directive('myCustomer', function() { return { restrict: 'E', scope: { customerInfo: '=info' }, templateUrl: 'my-customer-iso.html' }; });
<div ng-controller="Controller"> <my-customer info="naomi"></my-customer> <hr> <my-customer info="igor"></my-customer> </div>
controller里面绑定了 naomi 和 ingor 的值,html里面将directive
元素中的 info 绑定为 naomi 或 ingor. directive里面定义独立
上下文,将 customerInfo 绑定到元素里的
info. my-customer-iso.html 里面引用 customerInfo 的内容. 如果要
引用 <div bind-to-this="thing">
,
在directive里面应该用
=bindToThis
.如果isolate scope里的
属性名跟directive标签中的一样,则可以缩写为 XXX:
'='
.
注意:如同之前提到过的,isolate scope没有继承高级上下文的内容,在这
里面调用高级上下文中的变量,只能是空
template里面或者templateUrl引用的html里面的内容继承的是directive
的上下文
给予外部上下文访问能力.同时也意味着之前isolate scope 被遮盖了,无法访问.
angular.module('docsIsoFnBindExample', []) .controller('Controller', ['$scope', '$timeout', function($scope, $timeout) { $scope.name = 'Tobias'; $scope.hideDialog = function () { $scope.dialogIsHidden = true; $timeout(function () { $scope.dialogIsHidden = false; }, 2000); }; }]) .directive('myDialog', function() { return { restrict: 'E', transclude: true, scope: { 'close': '&onClose' }, templateUrl: '<div class="alert"><a href class="close" ng-click="close()">×</a><div ng-transclude></div></div>' }; });
<div ng-controller="Controller"> <my-dialog ng-hide="dialogIsHidden" on-close="hideDialog()"> Check out the contents, ! </my-dialog> </div>
点击动作为ng-click="close()"
,而
close
动作被绑定到 &onClose
这个directive. on-close
又在
html里面绑定到 hideDialog
函数上. 这个示例中显示了如何将函数表
达式传递给directive,让directive在一定时候选择执行这个函数.scope中
读取值用’@XX’,读取表达式结果用’=XX’,读取表达式但不执行用’&XX’.
不管是template还是templateUrl都是在directive的上下文下.当没有
isolate scope的时候,直接访问外部上下文.有isolate scope的时候,只能
访问isolate scope.这个时候html里面元素的内容,比如
<AA>blabla</AA>
里面的 blabla 是被忽略/覆盖的.
当在template中加入一个含有transclude的元素的时候,html里面的
blabla 会被显示在这个元素之中,而这个含有transclude的元素的内容
却不会被显示,同时html里面的表达式无法访问isolate scope.
... return { template: <div>CONTENT IN JAVASCRIPT (DIRECTIVE SCOPE (OUTER/ISOLATE SCOPE))</div><div ng-transclude>CONTENT FROM HTML (OUTER SCOPE)</div> }
在html里面用的时候
在JS里面使用一个叫做 filter 的过滤器
angular.module('FilterInControllerModule', []). controller('FilterController', ['filterFilter', function(filterFilter) { this.array = [ {name: 'Tobias'}, {name: 'Jeff'}, {name: 'Brian'}, {name: 'Igor'}, {name: 'James'}, {name: 'Brad'} ]; this.filteredArray = filterFilter(this.array, 'a'); }]);
创建过滤器
angular.module('myReverseFilterApp', []) .filter('reverse', function() { return function(input, uppercase) { input = input || ''; var out = ""; for (var i = 0; i < input.length; i++) { out = input.charAt(i) + out; } // conditional based on optional argument if (uppercase) { out = out.toUpperCase(); } return out; }; })
$filter
服务通过名字调用相应的过滤器.
学习AngularJS入门教程, 英文版
在HTML里面用np-app来定义Angular的上下文,可以用
ng-app="AAA"
来制定一个名字空间.JS中
通过这个名字来获取HTML里的上下文 var AAA = angular.module('AAA',
[]);
.
在HTML中Angular的范围内,用
ng-controller="BBB"
标记一个div,在里
面用一些Angular命令(ng-repeat)和表达式设定显示方式.在JS里面用
AAA.controller('BBB',function ...)
给Angular表达式中的域赋值.
AAA 中的 $scope$
是AAA的根作用域,控制器的作用域则在 BBB 之中.也
就是说,如果在AAA下面有多个 controller, controller X 的数据绑定
在 controller Y 下是无效的. AngularJS作用域文档.
略:AngularJS的开发者倾向于使用 Jasmine 行为驱动开发(BBD)框架进行 测试.
写在后面,对angular.module是所有JS中创建,注册和获取模组的地方.HTML里 面通过ng-app=”AA”来绑定模组AA的上下文
建立输入表单 <input
ng-model="query">
,修改 ng-repeat
为
<li ng-repeat="x in
list|filter:query">
.只要在输入框里输入文本,就会自动
地修改列表,只显示包含这个文本的项目.这代表两点:1. 数据绑定同名变量
query,值的同步是实时的; 2.filter过滤器,生成新数组,自动更新视图.
略:Angular端到端测试.
在HTML里面添加选择器
<select ng-model="orderProp"> <option value="name">Alphabetical</option> <option value="age">Newest</option> </select>
同时在迭代器中加入条件
<li ng-repeat="phone in phones | filter:query |
orderBy:orderProp">
JS中给每个数据加上 age 这个属性,同时设定
$scope.orderProp = 'age'
设定选择器
的默认值.
select
的元素和 orderProp
之间建立了双向绑定.
通过 $http.get(AA/BB.json)
来读取json文件. $http 是AngularJS提
供的内建服务,通过依赖注入DI子系统.
phonecatApp.controller('PhoneListCtrl', ['$scope', '$http', function($scope, $http) { $http.get('phones/phones.json').success(function(data) { $scope.phones = data; }); }]);
<ul class="phones"> <li ng-repeat="phone in phones | filter:query | orderBy:orderProp" class="thumbnail"> <a href="#/phones/" class="thumb"><img ng-src=""></a> <a href="#/phones/"></a> <p></p> </li> </ul>
用 $routeProvider
重定向url,把某个url路径定向到某个HTML模板上去,
并指定相对应的controller. index.html 中定义一个域,加载所有JS文件.加
载的 app.js 设定重定向规则.
phonecatApp.config(['$routeProvider', function($routeProvider) { $routeProvider. when('/phones', { templateUrl: 'partials/phone-list.html', controller: 'PhoneListCtrl' }). when('/phones/:phoneId', { templateUrl: 'partials/phone-detail.html', controller: 'PhoneDetailCtrl' }). otherwise({ redirectTo: '/phones' }); }]);
:phoneId
是从将URL路径的对应部分作为变量.这个变量通过
$routeParams
来访问.
最后,在index.html里面加上 <div ng-view></div>
来为当前路由把对应
视图加上去.
08更多模板 之中用 $http 加载不同的json文件来根据phoneId生成每 部手机的细节说明网页.
建立自己的 filter FF,实现更复杂的规则,然后像之前使用 | filter:query
一样用类似pipe的语法把输出传给自己的filter | FF
.filter并不限定在
ng-repeat 中使用.
angular.module('phonecatFilters', []).filter('checkmark', function() { return function(input) { return input ? '\u2713' : '\u2718'; }; });
AngularJS内置过滤器 uppercase, json, date, date:”MM/dd/yyy@ h:mma”.
通过JS在上下文中定义函数,通过 ng-click
来设定点击事件,调用函数.
phonecatControllers.controller('PhoneDetailCtrl', ['$scope', '$routeParams', '$http', function($scope, $routeParams, $http) { $http.get('phones/' + $routeParams.phoneId + '.json').success(function(data) { $scope.phone = data; $scope.mainImageUrl = data.images[0]; }); $scope.setImage = function(imageUrl) { $scope.mainImageUrl = imageUrl; } }]);
$scope.setImage
就是绑定的函数.
<img ng-src="" class="phone"> ... <ul class="phone-thumbs"> <li ng-repeat="img in phone.images"> <img ng-src="" ng-click="setImage(img)"> </li> </ul>
点击图片的时候调用函数,把 mainImageUrl
的值设定为当前图片.
依赖为 angular-resource/angular-resource.js.一般我们的JS相应功能 命名为 XXXService.首先注册 Service 模块.
var phonecatServices = angular.module('phonecatServices', ['ngResource']); phonecatServices.factory('Phone', ['$resource', function($resource){ return $resource('phones/:phoneId.json', {}, { query: {method:'GET', params:{phoneId:'phones'}, isArray:true} }); }]);
建立一个 Phone 工厂,用 $resource 来取得数据.相应的controller改
为$scope.phone =
Phone.get({phoneId:$routeParams.phoneId})
和
$scope.phones = Phone.query()
.
query
方法是自己声明和定义的 query: {method:'GET', params:{phoneId:'phones'}, isArray:true}
.
TODO
更多的教程 开发指南 和 Cookbook. 推荐使用AngularJS种子项目来引导开发新项目
JavaServer Page(JSP)是一种动态网页技术标准.在HTML中插入Java片段 (Scriptlet),以.jsp为文件后缀.在接到客户端请求的时候,服务器根据XXX.jsp文 件生成XXXServlet.java文件,然后编译运行,最后返回结果给客户端.
Servelet是一种 独立于平台和协议 的 服务器端 的Java应用程序,生 成动态页面.为HTTP客户端跟HTTP服务器程序的中间层.好处是快(比CGI快), 每个用户请求被激活成程序中的一个线程.Servlet是纯粹的Java,jsp介于 html和java质检,有助于美工人员来设计界面.
既然上一节提到了CGI,这里稍微展开一下.Common Gateway Interface (CGI)是一种利用程序的标准输入,输出流来完成HTTP通信的方式.HTTP本身 是文本协议,请求是由服务器(Apache)处理,而应用程序可能由各种语言来实 现.CGI则将这个文本协议以标准输入流的方式传递给服务器端的CGI程序. 与之类似的还有WSGI,原理类似,但是是Python专用的协议.对于实现了 call 的对象,将 request 经过包装作为参数传入,同时也返回进过 包装的 response.
Apache和Tomcat都是提供服务器功能的软件.Apache可以使用 PHP,CGI,Perl,Tomcat在Apache后端提供对Java的支持.Apache+Tomcat处理 JSP的时候,Apache只是转发.Apache是Web服务器,Tomcat是应用服务器,是 Servlet容器,可以单独运行.
Tomcat是Servlet的容器,可以独立运行,也可以作为现有服务器的附加(支持 Apache,IIS,Netscape).配置Tomcat的时候选择运行模式.
Java后端负责业务逻辑,数据持久化(数据库),安全等等.目前接触到的内容有
账号注册登录管理,本地化设置等等.将整个后端当做一个应用,那么在主目录
下有一个 Application.java
类作为整个应用的入口.这个类最基本的应用需要
@ComponentScan
和 @EnableAutoConfiguration
两个标签来声明自动配
置和扫描组件;同时在 main
函数里实例化一个 SpringApplication
对象
并调用对象函数 run()
就行了. 目前的程序里还用
app.setAdditionalProfiles
在没有命令行参数的时候默认加载dev配置.
SpringApplication
会自动搜索以下路径来寻找
application.properties
作为配置文件.
/config
子文件夹
classpath:/config
classpath
同时,SpringApplication还支持 YAML 文件,即 applicaiton.yml.
函数 SpringApplication.setAdditionalProfiles("dev")
则会额外加载
application-dev.yml (或者 application-dev.properties), 而不是
dev.yml.
当我们有类继承了EnvironmentAware这个接口的时候,框架会自动调用这个类
的 setEnvironment
方法,将一个 Environment
对象交给这个类的对象
从而使这个类获得配置文件中提供的参数. LocaleConfiguration
和
MailConfiguration
两个类展示了相关的编程方法.
public class MailConfiguration implements EnvironmentAware { public void setEnvironment(Environment environment) { this.propertyResolver = new RelaxedPropertyResolver(environment, ENV_SPRING_MAIL); } }
另一种方法是直接 @Inject
一个 Environment
的对象.
public class MailService { @Inject private Environment env; }
<–这种方法跟之前所介绍的有什么区别,暂时不知道.–>
LocalResolver
和 Interceptor
. 在接受请求的时候, Interceptor
拦截请求中特定的参数值,并且调用 LocalResolver
的函数来进行相关渲
染处理.在 LocaleConfiguration
中,龙伟权通过定义Bean来指定特定的
LocalResolver
, 即 AngularCookieLocaleResolver
.
问题:
LocaleConfiguration
和 ThymeleafConfiguration
都有方法返回
MessageSource
, MailService
里面貌似引用的是
LocaleConfiguration
返回的对象,为什么?
对象的关联问题.Mongodb本身作为NoSql的典范,内部以文件和类似json的数
据形式来储存数据.Mongodb中一个Document可以看做关系数据库中的一行数
据,一个Repository可以看做关系数据库中的一个表(table).目前我们有三个
标记为 @Document
的类 PersistentToken,Task,User
和对应的
Repository
类.
控制器类都在org.team.drill.web下面,统一以 XXXResource 命名.
AccountResource
UserResource
登录功能是在 SecurityConfig 中间接实现的.
Service 里面通过 SecurityContextHolder 来获取当前用户.
位于 web.dto 包下面的 UserDTO
类型,似乎是用来作为JS交互的数据
对象?但是内容上跟 domain 里的 User
差不多,用来保证安全.在更复
杂的应用环境中有用.
不论是Service还是Resource,都是靠 SecurityUtils.getCurrentLogin()
来获取当前用户的.那么问题来了,SecurityUtils如何知道当前用户是谁?
public final class SecurityUtils { private SecurityUtils() { } /** * Get the login of the current user. */ public static String getCurrentLogin() { SecurityContext securityContext = SecurityContextHolder.getContext(); Authentication authentication = securityContext.getAuthentication(); UserDetails springSecurityUser = null; String userName = null; if(authentication != null) { if (authentication.getPrincipal() instanceof UserDetails) { springSecurityUser = (UserDetails) authentication.getPrincipal(); userName = springSecurityUser.getUsername(); } else if (authentication.getPrincipal() instanceof String) { userName = (String) authentication.getPrincipal(); } } return userName; } }
SecurityContextHolder
包含安全上下文信息,并且默认为/ThreadLocal/,即对于一个线程采用一
个统一的安全上下文信息.同时,=SecurityContextHolder= 中还储存了当前
与App进行交互的主体(principal)的详情,用 Authentication
对象来表
示.上面的代码示例中体现的是获取当前用户信息的标准用法.
<–spring应用内部线程到底是如何管理 的?–>
UserDetails
在以上代码片段中我们可以看到一个特殊的类 UserDetails
,在获取当前主
体的时候,返回的可能是一个 String 或者一个 UserDetails 的对象.那
么这个/UserDetails/到底是什么东西,为什么它的方法 getUsername()
刚
好能包含我们的 User
类所需的 login?
UserDetails 是一个接口,包含了 getPassword()
, getUsername()
等
一系列方法. 在 UserDetails 中加入我们定义的 login,这是靠实现
UserDetailsService
接口来做到的. 这个接口只包含一个方法,这个方法
接受一个 String 参数,并返回一个实现了 UserDetails
接口的对象.目
前在org.team.drill.security包中的 UserDetailsService
类里面,龙伟
权实现了这个方法,并且将 org.team.drill.domain.User
的 login 包
装到实现了 UserDetails 接口的
org.springframework.security.core.userdetails.User
类中.
到这里,我试着把 SecurityContextHolder 到 UserDetails 串起来. 个
人认为,在通过
SecurityContextHolder.getContext().getAuthentication().getPrincipal()
来获取 UserDetails 的时候,spring通过调用实现了
org.springframework.security.core.userdetails.UserDetailsService
接口的对象的方法来尝试获取 UserDetails, 这个方法本身接受一个
String参数. 又绕回来了,这个String参数是
从哪来的?依靠http协议?还是session?
因为某种原因,
getPrincipal 方法返回的是一个 Object,那么实际上给出了一种可能性,就
是通过某个实现了 UserDetails 的类使用额外的业务逻辑.我们可以创建
一个实现了 UserDetals 的类 A,在 UserDetailsService 中返回 A
的对象,然后将 getPrincipal() 返回的对象强制转换为 A 并调用相关
方法.当然,强制转换类型总是有风险的.
GrantedAuthority
Authentication
提供两个重要的方法,第一个是上面说过的
getPrincipal,另一个则是 getAuthorities .它返回一个
GrantedAuthoriy 对象数组.顾名思义,这是用来管理用户权限的.在
UserDetailsService 里面进行加载.目前drill中加载的是
SimpleGrantedAuthority
数组,每个 SimpleGrantedAuthority 里面包
含一个字符串,代表某种权限(自定义于
org.team.drill.security.Authority
中).
用 random.nextBytes
来产生 PersistentToken 的序列号,不怕重复?
虽然16位Byte确实已经很大(10亿的4次方).
In this article, I will introduce the installation and basic use of MPI framework through octave. Matlab is widely used in many scientific domains as it enables the developers focus on the algorithms other than programming skills. However, the support of parallelism computing from Matlab is limited and expensive, especially for computing on clusters. Currently, two parallelism frameworks are widely used: MPI (Message Passage Interface) and MapReduce. Hadoop, which is an Java implementation of MapReduce, does not currently support Matlab or Octave. Therefore, I choose the MPI + Octave as a parallelism solution for Matlab users.
brew install open-mpi
sudo apt-get install openmpi-bin openmpi-common
openmpi-checkpoint libopenmpi-dev libopenmpi-X.X
. However,
the OpenMPI installed through this method does not work with
the octave-mpi package due to a library linkage problem. A
fast cue is compiling and installing it manually with specific
configurations.
Download the openmpi package openmpi-1.8.3.tar.gz.
tar xzvf openmpi-1.8.3.tar.gz cd openmpi-1.8.3 ./configure --dsiable-dlopen make sudo make install
brew tap homebrew/science
brew install gcc
brew install octave
sudo apt-get install octave
putenv ("PATH", ["path/to/mpic++/:" getenv("PATH")]) % optional pkg install -forge mpi % install mpi from internet or pkg install PATH/TO/PACKAGE % install mpi from downloaded package
apt-get
install octave-openmpi-ext
. As the pakcage manager could
automatically install the OpenMPI package, which does not work
with the octave-mpi actually, we suggest do NOT use this
method.
mpirun [OPTIONS] octave -q --eval '[octave code, function or script name]'
system ("mpirun octave -q --eval '[octave code, function or script name]");
/usr/share/doc/octave-mpi/examples/
/usr/share/octave/packages/mpi-1.2.0/
If you install the package under user’s home folder, you can
find exmaple codes under $HOME/octave/packages/mpi-1.2.0/
. To
test the example inside octave, run command
[info res] = system ("mpirun octave -q --eval 'pkg load mpi; helloworld
()'");
. The displayed string of the funciton, e.g. helloworld()
, will return by this invokation. Therefore your can embed it in another
serial computing octave script context (use str2num()).
--hostfile
indicate the file
that contains the hostnames of cluster; -np
number of
processes to run.
The interface provided by octave mpi package is a subset of all MPI functions.
Example 1, basic example about initializing the environment, send and receive messages/variables. Basically, these functions are enough to create the parallelism program.
MPI_Init(); communicator = MPI_Comm_Load("a label string") % one can use multiple communicators indicated by different label my_rank = MPI_Comm_rank(communicator) % rank is the indicator(id) of current process p = MPI_Comm_size(CW) % the size of a communicator is the number of processes handled by the communicator [info] = MPI_Send(value, ranks, TAG, communicator); % value: a octave variable % ranks: a vector that contains the id of destination processes, can be an integer % TAG: an integer to identify the message (when there are many messages sent in the same context) % info: an integer that indicates the success of failure state of process [value, info] = MPI_Recv(source, TAG, communicator) % source: the id the process that send this message % receive from any process if source is -1 % process starts from 0 and process 0 is usually used as master process % value: received variable MPI_Finalize()
Process sequence control functions:
MPI_Barrier(communicator)
will block the process until
all the processes reached this location.
MPI_Probe(RANK, TAG, COMM)
test if process [RANK] send
a message with TAG from communicator COMM. Block the process if
no message arrive.
MPI_Iprobe(RANK, TAG, COMM)
test if process [RANK] send
a message with TAG from communicator COMM. NOT block the process
if no message arrive.
If you run the octave with MPI, you will find multiple octave
processes in the process monitor. Once you run your MPI program,
multiple clones of your program are craeted. Each of them is a
full clone, which means the content before MPI_Init
or
MPI_Finalize
will be run in each process. You can test it by
putting some random function before MPI_Init
and see the
result. So, if you have a complex application and only want to use
MPI to accelerate a specific function, you should call the MPI
program as a independent application and get the result in
indirect way. For example, you can write the result into a file
and read it in your main application. But that would require a
distributed file system, which is out of the range of this article.
The MPI interface for C language is much more powerful than that for octave. But basically, it follows the same procedure: initializing MPI environment (world), communication with process id and message tag. I recommend the tutorial from [https://computing.llnl.gov/tutorials/mpi/] as a learning material. It provides many exercises for basic features of MPI in C.
Compile C source file using the command below, for more options, please check the document of the version you installed.
mpicc [YOUR_C_SOURCE_FILE]
MPI_Send(buffer,count,type,dest,tag,comm)
MPI_Recv(buffer,count,type,source,tag,comm, status)
&var
.
MPI_ISend(buffer,count,type,dest,tag,comm,request)
MPI_IRecv(buffer,count,type,source,tag,comm,request)
First, I have to clarify the blocking/non-blocking concept in this
context. It is different to the block/non-block in multiple
thread programming. Blocking means the application will not
continue until the data in the application buffer is completely
sent out (through internet protocol). Non-blocking means the
application continue without waiting. Pay attention, it wait
until the data is sent out, not received by other
processes. And I think that is why tag is necessary in both
cases, because if the application block until data received,
the tag is not necessary and the application becomes
sequential. Besides, you cannot put receive before send because
it is blocked.
What is the benefit of non-blocking? You don’t have to wait for the receiving of data. In the example code below, we can see that processes receive data before sending them.
#include "mpi.h" #include <stdio.h> main(int argc, char *argv[]) { int numtasks, rank, next, prev, buf[2], tag1=1, tag2=2; MPI_Request reqs[4]; MPI_Status stats[4]; MPI_Init(&argc,&argv); MPI_Comm_size(MPI_COMM_WORLD, &numtasks); MPI_Comm_rank(MPI_COMM_WORLD, &rank); prev = rank-1; next = rank+1; if (rank == 0) prev = numtasks - 1; if (rank == (numtasks - 1)) next = 0; MPI_Irecv(&buf[0], 1, MPI_INT, prev, tag1, MPI_COMM_WORLD, &reqs[0]); MPI_Irecv(&buf[1], 1, MPI_INT, next, tag2, MPI_COMM_WORLD, &reqs[1]); MPI_Isend(&rank, 1, MPI_INT, prev, tag2, MPI_COMM_WORLD, &reqs[2]); MPI_Isend(&rank, 1, MPI_INT, next, tag1, MPI_COMM_WORLD, &reqs[3]); { do some work } MPI_Waitall(4, reqs, stats); MPI_Finalize(); }
Groups can be destroyed and created during execution. One process can be in multiple groups.
After the installation and the configuration of jekyll along with the org-mode. I tried to find some configurations to facilitate the creation of org files (create file with date in front of file name, insert heads, etc.) I did not realize the existence of relative emacs packages in elpa and asked google for solutions. My inefficient search key-words let the google returns undesired answer: the octopress. Octopress is based on the jekyll, but provide richer blog settings. Finally, I migrated to octopress and use it to manager my github pages.
Installation of octopress is little complex since it requires low version ruby.
You can following the instructions on the official manual.
cd # go to home git clone git://github.com/sstephenson/rbenv.git .rbenv git clone git://github.com/sstephenson/ruby-build.git .rbenv/plugins/ruby-build
Add two lines into the end of shell profile (.zshrc in my case).
export PATH=$HOME/.rbenv/bin:$PATH eval "$(rbenv init -)"
Then reload the profile by source ~/.zshrc
or resume the
terminal.
The installation of rbenv and ruby-build can be replaced by
brew update brew install rbenv brew install ruby-build
Attention: the shell profile must be modified with both installation methods.
Then install ruby 1.9.3, which is required by octopress.
rbenv install 1.9.3-p0 rbenv local 1.9.3-p0 rbenv rehash
My setup experience is basically the same with the official
manual. However, after the installation, I replaced the octopress
directory to ~/WorkSpace/xiaoliuai.github.io
because it actually
becomes the repository of my github pages (magnificent!). So I
suggest to clone the octopress git repository into the directory
with the same name of the repository of your github pages at the
beginning.
(_I suggest to clone shallow copy with option_ depth
)
git clone git://github.com/imathis/octopress.git octopress
cd octopress
gem install bundler rbenv rehash # If you use rbenv, rehash to be able to run the bundle command bundle install
(_I suggest to ignore this step and install the 3rd-party theme directly_)
rake install
The octopress puts the generated web site, including the pages,
layouts, images, etc. into the public
folder for local preview. The source
folder contains all the source files used to generate the site,
including the layout HTML files, Java-scripts, style-sheets,
etc. It can be seemed as a hyper _posts
folder that contains
source files more than blog HTML files. The familiar _posts
folder used in jekyll is under this folder. Let’s start blogging.
First, go into the octopress folder, type rake
new_post["title"]
, octopress will ask you to give a blog tile,
then creates a markdown
file into the _posts
folder with
jekyll format (concatenate date and the title as the file name).
rake new_post["My first blog"] # create source/_posts/2014-10-28-my-first-blog.markdown
This file should be edited respect to the jekyll protocol.
Second, ran rake generate
to generate the site in public
and then ran rake preview
to mount a web-server at
http://localhost:4000.
Octopress integrated commands to let you deploy the generated site to the github.
rake setup_github_pages
This command will ask you for a URL of the Github repo, then it
will store your Github pages repository URL as the origin
remote and change the octopress remote to octopress. The
directory of octopress becomes the repository of the source
branch of the origin remote (your Github Pages repo). The
_deploy
directory, becomes the master branch of the origin
remote.
Run command:
rake deploy
will update the files in public
to _deploy
, then push the
changes to the master branch of the origin remote. To keep your
settings and sources tracked, run
git add .
git commit -m'Your message'
git push origin source
to push the contents in octopress directory to the source branch of the origin remote.
One thing confusing is that one repository (master branch) is
under another repository. But the directory _deploy
is in the
gitignore
of source branch, hence solve the self-contain
problem.
There are lisp packages in melpa that provide smart org-jekyll edit
functions.The great advantage of this package is that it generates
blogs with respect to the entries with blog
tag. In detail, it
generates HTML files with proper time stamp before the file
names. Therefore you don’t have to take care about the file names (you
have to manage the org file name if you use other tools). Note you
have to specify the time stamp in property of entry.
This package does only export the section names, modify the elisp
script following the issue to export the contents. Use
org-jekyll-export-blog
to export the blog HTML files.
;; change (org-map-entries (lambda () (org-jekyll-export-entry project) "blog|BLOG")))) ;; to (org-map-entries (lambda () (org-show-subtree) (org-jekyll-export-entry project)) "blog|BLOG"))))
The package can be installed from elpa, but it does not support the
tags. Hence, I modified the ox-jekyll.el file, duplicated all the
variables and expressions contain categories
and then replace
them by tags
. After reopening emacs, it successfully exports the
tags into HTML files under _posts.
标准的org-mode模式在导出为jekyll的HTML片段的时候,有一些导出格式需要 相应的css或者header支持. 不兼容列表
[1/4]
[X]
tags
[ ]
functions of contents in setupfile.org
[ ]
org-octopress generate publish
folder.
[ ]
In org-jekyll, YAML markup is mentioned many times. I have to
study into it when I have time.
This is my first blog that is generated using org-mode, jekyll and is published on GitHub. I made this page following the instructions on org-jekyll.
Note that the built-in org-mode in emacs might be outdated, the best
solution is to remove the built-in org-mode and reinstall it from
the package manager via M-x list-packages
. Jekyll can be installed
through RubyGem. On my OSX, the ruby and gem are preinstalled, so
gem install jekyll
works for me.
As my final goal is posting the blog to GitHub, the working folder
of my blog is xiaoliuai.github.io
.
jekyll new xiaoliuai.github.io cd xiaoliuai.github.io jekyll serve
Then you can see the portal page by opening localhost:4000
in
your web browser.
Now I would like to introudce the directory hierarchy of jekyll. Since I will move to octopress, which is a wrapper of jekyll with different directory setting, I just introduce the essential part to make the things work.
xiaoliuai.github.io/ -- _posts/ -- _sites/ ...
Two main directories need special attention, that are,
_posts
and _sites
. The first folder contains the source files,
which are markdown file, html file, etc. as input for jekyll; the
second folder contains the generated static server pages. All the
project should be uploaded to github to be displayed where the
index.html
file is the portal of web site. The GitHub Pages
understand this directory configuration and will automatically display
the contents in _sites
.
("blog-org" :base-directory "~/Emacs/org/blog" :base-extension "org" :publishing-directory "~/WorkSpace/xiaoliuai.github.io/" :recursive t :publishing-function org-html-publish-to-html :headline-levels 4 :html-extension "html" :body-only t ;; Only export section between <body> </body> :with-toc nil ;; don't export toc, it clobbers the YAML front matter ) ("blog-static" :base-directory "~/Emacs/org/blog" :base-extension "css\\|js\\|png\\|jpg\\|gif\\|pdf\\|mp3\\|swf\\|gz\\|txt\\|el" :publishing-directory "~/WorkSpace/xiaoliuai.github.io/" :recursive t :publishing-function org-publish-attachment ) ("blog":components ("blog-org" "blog-static"))
With this org-mode configuration, you have to create a folder _posts
under the ~/Emacs/org/blog
, then org-mode will export the folder and
html files into the right place.
#+STARTUP: showall indent #+STARTUP: hidestars #+BEGIN_HTML --- layout: default title: Xiao Liu's First Blog excerpt: First blog with org-mode, jekyll. categories: - org --- #+END_HTML
The two lines with comments form the main differences to ordinary html exportation. Jekyll will read the body of html files generated by org-mode and add it’s own heads to specify the styles. Org-mode will export the table of contents (TOC) at the beginning of exported html file, hence mask the YAML Front Matter. Disable the TOC can sovle this problem. However, there might be other solutions to change the position of TOC.
cd xiaoliuai.github.io jekyll build jekyll serve
The simplest personalization of the blog site is editting
_config.yml
under the project directory. Following the attribute
names in this file, you can easily understand how to specify the blog
name, description, email address, etc.